[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:39:50 dist-test-slave-h6n0 krb5kdc[1498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:39:50 dist-test-slave-h6n0 krb5kdc[1498](info): set up 2 sockets
Aug 07 00:39:50 dist-test-slave-h6n0 krb5kdc[1498](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 00:39:51.716464 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:39:51 dist-test-slave-h6n0 krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:39:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:39:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:39:51.851742 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:33441
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:33441
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:51.944262 1518 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:51.944551 1518 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:51.944594 1518 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:51.946007 1518 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:39:51.946079 1518 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:51.946107 1518 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:51.946139 1518 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:39:51.946163 1518 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:39:51.948144 1518 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:33441
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:33441
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1518
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:51.948750 1518 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:51.949128 1518 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:51.955742 1526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:51.955751 1525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:51.955874 1528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:51.955760 1518 server_base.cc:1034] running on GCE node
I20260807 00:39:51.956251 1518 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:51.956633 1518 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:51.957830 1518 hybrid_clock.cc:648] HybridClock initialized: now 1786063191957814 us; error 104 us; skew 500 ppm
Aug 07 00:39:51 dist-test-slave-h6n0 krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063191, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:51.960549 1518 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:39:51.961118 1518 webserver.cc:460] Webserver started at http://127.1.116.190:42049/ using document root <none> and password file <none>
I20260807 00:39:51.961457 1518 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:51.961536 1518 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:51.961728 1518 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:51.963166 1518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "c9242a9da27f40a0a55926b969e0d8d4"
format_stamp: "Formatted at 2026-08-07 00:39:51 on dist-test-slave-h6n0"
server_key: "0ecfc3d71506e29e22bb33f94aa6146c"
server_key_iv: "cc37a4645bfd96579d4a1031f631cb16"
server_key_version: "encryptionkey@0"
I20260807 00:39:51.963811 1518 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "c9242a9da27f40a0a55926b969e0d8d4"
format_stamp: "Formatted at 2026-08-07 00:39:51 on dist-test-slave-h6n0"
server_key: "0ecfc3d71506e29e22bb33f94aa6146c"
server_key_iv: "cc37a4645bfd96579d4a1031f631cb16"
server_key_version: "encryptionkey@0"
I20260807 00:39:51.965569 1518 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:39:51.966760 1535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:51.967067 1518 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:51.967196 1518 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "c9242a9da27f40a0a55926b969e0d8d4"
format_stamp: "Formatted at 2026-08-07 00:39:51 on dist-test-slave-h6n0"
server_key: "0ecfc3d71506e29e22bb33f94aa6146c"
server_key_iv: "cc37a4645bfd96579d4a1031f631cb16"
server_key_version: "encryptionkey@0"
I20260807 00:39:51.967285 1518 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:51.986070 1518 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:51.988404 1518 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:51.988580 1518 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:51.994211 1518 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:33441
I20260807 00:39:51.994280 1616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:33441 every 8 connection(s)
I20260807 00:39:51.994807 1518 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:39:51.996263 1618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:51.999253 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1518
I20260807 00:39:51.999387 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:39:51.999858 1618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4: Bootstrap starting.
I20260807 00:39:52.000092 1490 external_mini_cluster.cc:1428] Setting key 24e5e9fd3f2cc8b4089119d3608c3e46
I20260807 00:39:52.001125 1618 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:52.001673 1618 log.cc:824] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:52.002688 1618 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4: No bootstrap required, opened a new log
I20260807 00:39:52.005270 1618 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9242a9da27f40a0a55926b969e0d8d4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33441 } }
I20260807 00:39:52.005472 1618 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:52.005513 1618 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9242a9da27f40a0a55926b969e0d8d4, State: Initialized, Role: FOLLOWER
I20260807 00:39:52.005673 1618 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9242a9da27f40a0a55926b969e0d8d4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33441 } }
I20260807 00:39:52.005777 1618 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:39:52.005815 1618 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:39:52.005872 1618 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:52.007414 1618 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9242a9da27f40a0a55926b969e0d8d4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33441 } }
I20260807 00:39:52.007550 1618 leader_election.cc:302] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9242a9da27f40a0a55926b969e0d8d4; no voters:
I20260807 00:39:52.007769 1618 leader_election.cc:288] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:39:52.007841 1631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:52.008129 1631 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [term 1 LEADER]: Becoming Leader. State: Replica: c9242a9da27f40a0a55926b969e0d8d4, State: Running, Role: LEADER
I20260807 00:39:52.008320 1618 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:39:52.008283 1631 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9242a9da27f40a0a55926b969e0d8d4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33441 } }
I20260807 00:39:52.008739 1631 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9242a9da27f40a0a55926b969e0d8d4. Latest consensus state: current_term: 1 leader_uuid: "c9242a9da27f40a0a55926b969e0d8d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9242a9da27f40a0a55926b969e0d8d4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33441 } } }
I20260807 00:39:52.008762 1632 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9242a9da27f40a0a55926b969e0d8d4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9242a9da27f40a0a55926b969e0d8d4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33441 } } }
I20260807 00:39:52.008839 1631 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:52.008886 1632 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4 [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:52.009346 1646 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:39:52.010442 1646 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:39:52.012836 1646 catalog_manager.cc:1347] Generated new cluster ID: e6df02f831304763be3a68dc6371ae96
I20260807 00:39:52.012909 1646 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:52.020787 1629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.001174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.1.116.190:33441)
0807 00:39:52.001553 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:52.001560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:52.001811 (+ 251us) server_negotiation.cc:407] Connection header received
0807 00:39:52.005242 (+ 3431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:52.005249 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:52.005301 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:52.005423 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:52.008185 (+ 2762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.009942 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.011640 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.012372 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.012509 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.016336 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:52.016342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:52.016342 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:52.016357 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:52.018747 (+ 2390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.019274 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.019277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.019277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.019376 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.019654 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.019655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.019655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.019889 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:52.020079 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:52.020336 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 00:39:52.020477 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":83,"threads_started":1}
I20260807 00:39:52.027019 1646 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:39:52.027608 1646 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:39:52.037974 1646 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9242a9da27f40a0a55926b969e0d8d4: Generated new TSK 0
I20260807 00:39:52.038131 1646 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:39:52.073659 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33441
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:52.190038 1661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:52.190263 1661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:52.190300 1661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:52.191520 1661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:52.191589 1661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:52.191636 1661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:39:52.193115 1661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:52.193476 1661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:52.193717 1661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:52.199177 1674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:52.199276 1670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:52.199178 1667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:52.199596 1661 server_base.cc:1034] running on GCE node
I20260807 00:39:52.199816 1661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:52.200088 1661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:52.201261 1661 hybrid_clock.cc:648] HybridClock initialized: now 1786063192201246 us; error 39 us; skew 500 ppm
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:52.204190 1661 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:39:52.204730 1661 webserver.cc:460] Webserver started at http://127.1.116.129:33463/ using document root <none> and password file <none>
I20260807 00:39:52.204958 1661 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:52.205025 1661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:52.205166 1661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:52.206169 1661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "ae405e458a5e4a1f8fe00be25b83b30c"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "7c9ada0adf498651c7f832b5837dc8b1"
server_key_iv: "d931c237202efdd34c1e38b1325e79a8"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.206580 1661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "ae405e458a5e4a1f8fe00be25b83b30c"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "7c9ada0adf498651c7f832b5837dc8b1"
server_key_iv: "d931c237202efdd34c1e38b1325e79a8"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.207983 1661 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:39:52.208928 1685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.209194 1661 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:52.209317 1661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "ae405e458a5e4a1f8fe00be25b83b30c"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "7c9ada0adf498651c7f832b5837dc8b1"
server_key_iv: "d931c237202efdd34c1e38b1325e79a8"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.209458 1661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:52.222477 1661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:52.225035 1661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:52.225174 1661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:52.225399 1661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:39:52.225941 1661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:52.226027 1661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.226054 1661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:52.226068 1661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.234722 1661 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:45269
I20260807 00:39:52.234912 1860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:45269 every 8 connection(s)
I20260807 00:39:52.235225 1661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:39:52.241439 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1661
I20260807 00:39:52.241554 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:39:52.241760 1490 external_mini_cluster.cc:1428] Setting key 56b0f020f563ac7bedd2189fa957e29b
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:52.247665 1629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.236290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:49893 (local address 127.1.116.190:33441)
0807 00:39:52.236375 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:52.236378 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:52.236573 (+ 195us) server_negotiation.cc:407] Connection header received
0807 00:39:52.237127 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:52.237128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:52.237145 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:52.237231 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:52.238373 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.239064 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.240431 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.240647 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.240725 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.244121 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:52.244123 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:52.244123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:52.244133 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:52.245926 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.246484 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.246485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.246485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.246588 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.246919 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.246920 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.246920 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.247033 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:52.247092 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:52.247474 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 00:39:52.247564 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:39:52.248021 1867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.236186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33441 (local address 127.1.116.129:49893)
0807 00:39:52.236423 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:52.236466 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:52.237015 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:52.237354 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:52.237358 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:52.238053 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:52.238252 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:52.238257 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.239172 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.239173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:52.240194 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:52.240199 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.240747 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.240748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:52.240976 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.241727 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:52.241736 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:52.244007 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:52.246034 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:52.246042 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:52.246044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:52.246390 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:52.246693 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:52.246694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:52.246695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:52.246842 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:52.247109 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:52.247116 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:52.247279 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 00:39:52.247401 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 00:39:52.247589 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":124,"threads_started":1}
I20260807 00:39:52.248811 1861 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33441
I20260807 00:39:52.248926 1861 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:52.249140 1861 heartbeater.cc:511] Master 127.1.116.190:33441 requested a full tablet report, sending...
I20260807 00:39:52.250011 1561 ts_manager.cc:194] Registered new tserver with Master: ae405e458a5e4a1f8fe00be25b83b30c (127.1.116.129:45269)
I20260807 00:39:52.251199 1561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:49893
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:39:52.269969 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33441
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:52.372350 1870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:52.372634 1870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:52.372674 1870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:52.373904 1870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:52.373974 1870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:52.374022 1870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:39:52.375658 1870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:52.375975 1870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:52.376266 1870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:52.382027 1882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:52.382064 1880 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:52.382061 1885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:52.382339 1870 server_base.cc:1034] running on GCE node
I20260807 00:39:52.382634 1870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:52.382925 1870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:52.384091 1870 hybrid_clock.cc:648] HybridClock initialized: now 1786063192384075 us; error 36 us; skew 500 ppm
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:52.386848 1870 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:39:52.387321 1870 webserver.cc:460] Webserver started at http://127.1.116.130:34637/ using document root <none> and password file <none>
I20260807 00:39:52.387584 1870 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:52.387655 1870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:52.387812 1870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:52.388950 1870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "ff4c649c375c4c56a15d14f81077b0bb"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "3abeada77af9eb7e529e1d9051a2db19"
server_key_iv: "a1e22f264922e1cc69e76708c2b869b8"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.389340 1870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "ff4c649c375c4c56a15d14f81077b0bb"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "3abeada77af9eb7e529e1d9051a2db19"
server_key_iv: "a1e22f264922e1cc69e76708c2b869b8"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.391016 1870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260807 00:39:52.392022 1897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.392347 1870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:52.392498 1870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "ff4c649c375c4c56a15d14f81077b0bb"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "3abeada77af9eb7e529e1d9051a2db19"
server_key_iv: "a1e22f264922e1cc69e76708c2b869b8"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.392580 1870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:52.406929 1870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:52.409375 1870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:52.409530 1870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:52.409749 1870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:39:52.410092 1870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:52.410145 1870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.410187 1870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:52.410214 1870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.421715 1870 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:39349
I20260807 00:39:52.421905 2072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:39349 every 8 connection(s)
I20260807 00:39:52.422149 1870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:52.429443 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1870
I20260807 00:39:52.429586 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:39:52.429819 1490 external_mini_cluster.cc:1428] Setting key 1094878d50d3c15478b437ba7b88f133
I20260807 00:39:52.433158 1629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.423189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:45135 (local address 127.1.116.190:33441)
0807 00:39:52.423289 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:52.423293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:52.423596 (+ 303us) server_negotiation.cc:407] Connection header received
0807 00:39:52.424091 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:52.424092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:52.424105 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:52.424169 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:52.425190 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.425741 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.426918 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.427116 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.427186 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.429610 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:52.429612 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:52.429612 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:52.429641 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:52.431476 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.431977 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.431978 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.431979 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.432049 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.432317 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.432318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.432318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.432481 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:52.432557 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:52.433015 (+ 458us) server_negotiation.cc:299] Negotiation successful
0807 00:39:52.433090 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:39:52.433470 2076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.423171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33441 (local address 127.1.116.130:45135)
0807 00:39:52.423450 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:52.423490 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:52.423981 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:52.424236 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:52.424240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:52.424839 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:52.425096 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:52.425101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.425835 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.425835 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:52.426821 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:52.426823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.427204 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.427205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:52.427313 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.427844 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:52.427852 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:52.429451 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:52.431577 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:52.431586 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:52.431588 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:52.431885 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:52.432128 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:52.432129 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:52.432129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:52.432243 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:52.432564 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:52.432570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:52.432797 (+ 227us) client_negotiation.cc:769] Sending connection context
0807 00:39:52.432975 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 00:39:52.433194 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":156,"threads_started":1}
I20260807 00:39:52.434165 2073 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33441
I20260807 00:39:52.434297 2073 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:52.434582 2073 heartbeater.cc:511] Master 127.1.116.190:33441 requested a full tablet report, sending...
I20260807 00:39:52.435086 1561 ts_manager.cc:194] Registered new tserver with Master: ff4c649c375c4c56a15d14f81077b0bb (127.1.116.130:39349)
I20260807 00:39:52.435482 1561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:45135
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:39:52.459735 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33441
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:52.565300 2080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:52.565557 2080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:52.565583 2080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:52.566897 2080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:52.566944 2080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:52.566973 2080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:39:52.568554 2080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33441
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:52.568840 2080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:52.569128 2080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:52.574993 2088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:52.575007 2093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:52.575057 2089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:52.575373 2080 server_base.cc:1034] running on GCE node
I20260807 00:39:52.575747 2080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:52.576097 2080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:52.577284 2080 hybrid_clock.cc:648] HybridClock initialized: now 1786063192577263 us; error 46 us; skew 500 ppm
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:52.580673 2080 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:39:52.581163 2080 webserver.cc:460] Webserver started at http://127.1.116.131:39347/ using document root <none> and password file <none>
I20260807 00:39:52.581406 2080 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:52.581476 2080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:52.581650 2080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:52.582886 2080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "913ed1893ad348d3a69ebe30b8efddbb"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "7a569553d57381763980cd137cfcbdcc"
server_key_iv: "bab0ac9b7a9a1c291c07182a7fa1271e"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.583280 2080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "913ed1893ad348d3a69ebe30b8efddbb"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "7a569553d57381763980cd137cfcbdcc"
server_key_iv: "bab0ac9b7a9a1c291c07182a7fa1271e"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.585052 2080 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 00:39:52.586032 2103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.586306 2080 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:52.586438 2080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "913ed1893ad348d3a69ebe30b8efddbb"
format_stamp: "Formatted at 2026-08-07 00:39:52 on dist-test-slave-h6n0"
server_key: "7a569553d57381763980cd137cfcbdcc"
server_key_iv: "bab0ac9b7a9a1c291c07182a7fa1271e"
server_key_version: "encryptionkey@0"
I20260807 00:39:52.586524 2080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:52.598232 2080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:52.600342 2080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:52.600514 2080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:52.600809 2080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:39:52.601135 2080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:52.601193 2080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.601229 2080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:52.601274 2080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:52.611968 2080 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:40341
I20260807 00:39:52.612027 2273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:40341 every 8 connection(s)
I20260807 00:39:52.612494 2080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:39:52.616609 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2080
I20260807 00:39:52.616726 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:39:52.616938 1490 external_mini_cluster.cc:1428] Setting key 507cbf79ff59ab5c13aae73956d697e6
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:52.623854 1629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.613664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:59193 (local address 127.1.116.190:33441)
0807 00:39:52.613727 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:52.613729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:52.613901 (+ 172us) server_negotiation.cc:407] Connection header received
0807 00:39:52.614658 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:52.614659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:52.614673 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:52.614740 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:52.615931 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.616503 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.618164 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.618449 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.618573 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.621200 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:52.621202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:52.621202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:52.621209 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:52.622489 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.623002 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.623002 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.623003 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.623056 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.623289 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.623290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.623290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.623402 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:52.623459 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:52.623718 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 00:39:52.623789 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 00:39:52.624122 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.613542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33441 (local address 127.1.116.131:59193)
0807 00:39:52.613733 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:52.613788 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:52.614485 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:52.614819 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:52.614823 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:52.615525 (+ 702us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:52.615821 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:52.615828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.616755 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.616757 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:52.618063 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:52.618068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.618597 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.618598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:52.618777 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.619404 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:52.619413 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:52.621074 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:52.622683 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:52.622689 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:52.622691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:52.622929 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:52.623123 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:52.623124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:52.623124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:52.623225 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:52.623468 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:52.623472 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:52.623595 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 00:39:52.623684 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 00:39:52.623838 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":89,"threads_started":1}
I20260807 00:39:52.624742 2274 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33441
I20260807 00:39:52.624837 2274 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:52.625062 2274 heartbeater.cc:511] Master 127.1.116.190:33441 requested a full tablet report, sending...
I20260807 00:39:52.625502 1561 ts_manager.cc:194] Registered new tserver with Master: 913ed1893ad348d3a69ebe30b8efddbb (127.1.116.131:40341)
I20260807 00:39:52.625921 1561 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:59193
I20260807 00:39:52.629694 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:39:52.657871 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.649021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53914 (local address 127.1.116.129:45269)
0807 00:39:52.649259 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:52.649265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:52.649286 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:39:52.649320 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:52.649323 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:52.649349 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:52.649435 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:52.650051 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.650906 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.651669 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.652027 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.652141 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.654494 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:52.654502 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:52.654502 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:52.654509 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:52.656318 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.656782 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.656785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.656785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.656847 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.657215 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.657216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.657216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.657371 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:52.657585 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:52.657695 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 00:39:52.657779 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":105,"threads_started":1}
W20260807 00:39:52.658511 1730 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53914
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[1498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:39:52.684877 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:52.676969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53924 (local address 127.1.116.129:45269)
0807 00:39:52.677061 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:52.677063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:52.677187 (+ 124us) server_negotiation.cc:407] Connection header received
0807 00:39:52.677272 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:52.677273 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:52.677288 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:52.677347 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:52.677986 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.678495 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.679356 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:52.679560 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:52.679622 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:52.682306 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:52.682307 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:52.682308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:52.682314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:52.683728 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.684151 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.684152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.684153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.684224 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:52.684426 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:52.684427 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:52.684427 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:52.684521 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:52.684620 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:52.684708 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 00:39:52.684781 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 00:39:52.685703 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1661
I20260807 00:39:52.697583 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1870
I20260807 00:39:52.710472 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2080
I20260807 00:39:52.720276 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1518
2026-08-07T00:39:52Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2339 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[2293](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[2293](info): set up 2 sockets
Aug 07 00:39:52 dist-test-slave-h6n0 krb5kdc[2293](info): commencing operation
krb5kdc: starting...
W20260807 00:39:53.975924 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:39:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:39:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:39:54.103505 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:42651
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:42651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:54.194510 2316 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:54.194842 2316 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:54.194881 2316 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:54.196069 2316 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:39:54.196151 2316 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:54.196178 2316 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:54.196210 2316 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:39:54.196234 2316 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:39:54.197783 2316 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:42651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:42651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2316
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:54.198084 2316 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:54.198390 2316 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:54.203730 2325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.203717 2322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.203717 2321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:54.204200 2316 server_base.cc:1034] running on GCE node
I20260807 00:39:54.204504 2316 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:54.204878 2316 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:54.206055 2316 hybrid_clock.cc:648] HybridClock initialized: now 1786063194206036 us; error 39 us; skew 500 ppm
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:54.216048 2316 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:39:54.216569 2316 webserver.cc:460] Webserver started at http://127.1.116.190:46603/ using document root <none> and password file <none>
I20260807 00:39:54.216820 2316 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:54.216897 2316 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:54.217056 2316 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:54.218225 2316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "9f942e63836f4cdd90486f23e167d660"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "4f8c406bc9099c5154f902ae7515cf87"
server_key_iv: "9b45dcdbc64d8296af3ece2e789934cc"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.218679 2316 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "9f942e63836f4cdd90486f23e167d660"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "4f8c406bc9099c5154f902ae7515cf87"
server_key_iv: "9b45dcdbc64d8296af3ece2e789934cc"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.220284 2316 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:39:54.221247 2335 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.221557 2316 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:39:54.221679 2316 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "9f942e63836f4cdd90486f23e167d660"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "4f8c406bc9099c5154f902ae7515cf87"
server_key_iv: "9b45dcdbc64d8296af3ece2e789934cc"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.221766 2316 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:54.240478 2316 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:54.242419 2316 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:54.242619 2316 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:54.247036 2316 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:42651
I20260807 00:39:54.247126 2428 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:42651 every 8 connection(s)
I20260807 00:39:54.247476 2316 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:39:54.248536 2430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:54.249401 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2316
I20260807 00:39:54.249501 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:39:54.249667 1490 external_mini_cluster.cc:1428] Setting key 65a66a41e323b67b7ed328845f3fe5ad
I20260807 00:39:54.251958 2430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660: Bootstrap starting.
I20260807 00:39:54.252691 2430 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:54.253042 2430 log.cc:824] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:54.253801 2430 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660: No bootstrap required, opened a new log
I20260807 00:39:54.256227 2430 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f942e63836f4cdd90486f23e167d660" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42651 } }
I20260807 00:39:54.256422 2430 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:54.256459 2430 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f942e63836f4cdd90486f23e167d660, State: Initialized, Role: FOLLOWER
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:54.256644 2430 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f942e63836f4cdd90486f23e167d660" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42651 } }
I20260807 00:39:54.256747 2430 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:39:54.256795 2430 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:39:54.256870 2430 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:54.257712 2430 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f942e63836f4cdd90486f23e167d660" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42651 } }
I20260807 00:39:54.257841 2430 leader_election.cc:302] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f942e63836f4cdd90486f23e167d660; no voters:
I20260807 00:39:54.258038 2430 leader_election.cc:288] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:39:54.258375 2430 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:39:54.258443 2437 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:54.258662 2437 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [term 1 LEADER]: Becoming Leader. State: Replica: 9f942e63836f4cdd90486f23e167d660, State: Running, Role: LEADER
I20260807 00:39:54.258870 2437 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f942e63836f4cdd90486f23e167d660" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42651 } }
I20260807 00:39:54.259567 2442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f942e63836f4cdd90486f23e167d660. Latest consensus state: current_term: 1 leader_uuid: "9f942e63836f4cdd90486f23e167d660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f942e63836f4cdd90486f23e167d660" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42651 } } }
I20260807 00:39:54.259670 2442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:54.259706 2441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f942e63836f4cdd90486f23e167d660" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f942e63836f4cdd90486f23e167d660" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42651 } } }
I20260807 00:39:54.259816 2441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660 [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:54.260088 2461 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:39:54.260891 2461 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:39:54.261490 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.250251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46410 (local address 127.1.116.190:42651)
0807 00:39:54.250422 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.250430 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.250504 (+ 74us) server_negotiation.cc:407] Connection header received
0807 00:39:54.251130 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.251136 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.251182 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.251329 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.251948 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.252978 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.253758 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.254225 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.254346 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.257177 (+ 2831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.257182 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.257182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.257198 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.259476 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.259930 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.259933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.259934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.260045 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.260367 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.260369 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.260369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.260724 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.260865 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.261148 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.261291 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":114,"thread_start_us":79,"threads_started":1}
I20260807 00:39:54.262892 2461 catalog_manager.cc:1347] Generated new cluster ID: 42dca69f642148b284b4db9e9b65c9bc
I20260807 00:39:54.262954 2461 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:39:54.269268 2461 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:39:54.270057 2461 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:39:54.275908 2461 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f942e63836f4cdd90486f23e167d660: Generated new TSK 0
I20260807 00:39:54.276096 2461 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:39:54.310899 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42651
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:54.396773 2476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:54.396991 2476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:54.397035 2476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:54.398169 2476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:54.398239 2476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:54.398286 2476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:39:54.399915 2476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:54.400152 2476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:54.400372 2476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:54.405640 2486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.405669 2481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.405669 2483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:54.405961 2476 server_base.cc:1034] running on GCE node
I20260807 00:39:54.406263 2476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:54.406514 2476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:54.407707 2476 hybrid_clock.cc:648] HybridClock initialized: now 1786063194407691 us; error 37 us; skew 500 ppm
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:54.410308 2476 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:39:54.410804 2476 webserver.cc:460] Webserver started at http://127.1.116.129:36671/ using document root <none> and password file <none>
I20260807 00:39:54.411048 2476 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:54.411118 2476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:54.411258 2476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:54.412233 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "8b15f7f899bfe929786846993a6163ce"
server_key_iv: "eb8c4336a4e38ed3ca5fe99529088ec7"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.412591 2476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "8b15f7f899bfe929786846993a6163ce"
server_key_iv: "eb8c4336a4e38ed3ca5fe99529088ec7"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.413933 2476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:39:54.414745 2499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.414969 2476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:54.415081 2476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "8b15f7f899bfe929786846993a6163ce"
server_key_iv: "eb8c4336a4e38ed3ca5fe99529088ec7"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.415166 2476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:54.430135 2476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:54.432060 2476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:54.432220 2476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:54.432439 2476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:39:54.432740 2476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:54.432797 2476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.432838 2476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:54.432865 2476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.439685 2476 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:44257
I20260807 00:39:54.439744 2668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:44257 every 8 connection(s)
I20260807 00:39:54.440008 2476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:54.447402 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2476
I20260807 00:39:54.447484 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:39:54.447670 1490 external_mini_cluster.cc:1428] Setting key a13fddd2b395c30352426cb3104b49e4
I20260807 00:39:54.451117 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.440748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:38103 (local address 127.1.116.190:42651)
0807 00:39:54.440867 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.440871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.441074 (+ 203us) server_negotiation.cc:407] Connection header received
0807 00:39:54.442287 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.442288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.442303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.442366 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.443141 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.443700 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.444806 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.445047 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.445152 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.447726 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.447728 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.447729 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.447737 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.449328 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.449947 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.449948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.449949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.450011 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.450360 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.450361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.450362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.450489 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.450615 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.450914 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.451012 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 00:39:54.451378 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.440689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42651 (local address 127.1.116.129:38103)
0807 00:39:54.440896 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.440965 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.442150 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.442447 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.442452 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.442798 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:54.443016 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.443020 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.443808 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.443810 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.444708 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.444710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.445166 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.445167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.445313 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.445880 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:54.445889 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:54.447622 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:54.449448 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:54.449458 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:54.449460 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:54.449853 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:54.450153 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:54.450154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:54.450154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:54.450277 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:54.450607 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:54.450615 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:54.450763 (+ 148us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.450891 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.451088 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":91,"threads_started":1}
I20260807 00:39:54.452023 2670 heartbeater.cc:347] Connected to a master server at 127.1.116.190:42651
I20260807 00:39:54.452124 2670 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:54.452322 2670 heartbeater.cc:511] Master 127.1.116.190:42651 requested a full tablet report, sending...
I20260807 00:39:54.452864 2367 ts_manager.cc:194] Registered new tserver with Master: bdbdf4a6e74c4d21b28aa6a60b0f8bf0 (127.1.116.129:44257)
I20260807 00:39:54.453608 2367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:38103
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:39:54.475687 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42651
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:54.565754 2677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:54.566017 2677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:54.566056 2677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:54.567236 2677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:54.567306 2677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:54.567354 2677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:39:54.568976 2677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:54.569257 2677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:54.569502 2677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:54.574869 2686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.574885 2688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.574869 2690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:54.575307 2677 server_base.cc:1034] running on GCE node
I20260807 00:39:54.575529 2677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:54.575749 2677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:54.576918 2677 hybrid_clock.cc:648] HybridClock initialized: now 1786063194576908 us; error 32 us; skew 500 ppm
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:54.579706 2677 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:39:54.580240 2677 webserver.cc:460] Webserver started at http://127.1.116.130:37137/ using document root <none> and password file <none>
I20260807 00:39:54.580484 2677 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:54.580556 2677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:54.580722 2677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:54.581884 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "f441263682354fc08cb498decb3dcc61"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "caae455dee111f0eb79a433932ccf9a4"
server_key_iv: "6c1c4888620af52dbadc2f1fcf8d163a"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.582283 2677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "f441263682354fc08cb498decb3dcc61"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "caae455dee111f0eb79a433932ccf9a4"
server_key_iv: "6c1c4888620af52dbadc2f1fcf8d163a"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.584278 2677 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 00:39:54.585219 2698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.585469 2677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:54.585584 2677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "f441263682354fc08cb498decb3dcc61"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "caae455dee111f0eb79a433932ccf9a4"
server_key_iv: "6c1c4888620af52dbadc2f1fcf8d163a"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.585701 2677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:54.594491 2677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:54.597083 2677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:54.597306 2677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:54.597584 2677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:39:54.597956 2677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:54.598016 2677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.598055 2677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:54.598100 2677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.605911 2677 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:41169
I20260807 00:39:54.605983 2873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:41169 every 8 connection(s)
I20260807 00:39:54.606374 2677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:39:54.611431 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2677
I20260807 00:39:54.611531 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:39:54.611693 1490 external_mini_cluster.cc:1428] Setting key e0846f77c43b35249db0691318e6d38e
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:54.618248 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.607308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:38733 (local address 127.1.116.190:42651)
0807 00:39:54.607393 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.607397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.607638 (+ 241us) server_negotiation.cc:407] Connection header received
0807 00:39:54.608795 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.608797 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.608808 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.608868 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.609495 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.610029 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.611173 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.611371 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.611433 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.614267 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.614269 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.614270 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.614277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.616176 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.616793 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.616795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.616796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.616934 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.617325 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.617326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.617326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.617503 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.617611 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.618010 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.618127 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:39:54.618285 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.607293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42651 (local address 127.1.116.130:38733)
0807 00:39:54.607493 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.607535 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.608669 (+ 1134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.608938 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.608942 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.609208 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:54.609414 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.609418 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.610114 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.610115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.611060 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.611063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.611467 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.611468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.611638 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.612318 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:54.612328 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:54.614137 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:54.616283 (+ 2146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:54.616291 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:54.616293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:54.616694 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:54.617035 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:54.617036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:54.617037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:54.617185 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:54.617611 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:54.617618 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:54.617826 (+ 208us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.617947 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.618094 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260807 00:39:54.618960 2875 heartbeater.cc:347] Connected to a master server at 127.1.116.190:42651
I20260807 00:39:54.619065 2875 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:54.619275 2875 heartbeater.cc:511] Master 127.1.116.190:42651 requested a full tablet report, sending...
I20260807 00:39:54.619752 2367 ts_manager.cc:194] Registered new tserver with Master: f441263682354fc08cb498decb3dcc61 (127.1.116.130:41169)
I20260807 00:39:54.620174 2367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:38733
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:39:54.641853 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42651
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34127
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:54.728279 2882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:54.728467 2882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:54.728490 2882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:54.730159 2882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:54.730207 2882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:54.730234 2882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:39:54.731712 2882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34127
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42651
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:54.731894 2882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:54.732051 2882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:54.737545 2887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.737711 2890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:54.737545 2888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:54.737622 2882 server_base.cc:1034] running on GCE node
I20260807 00:39:54.738029 2882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:54.738330 2882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:54.739511 2882 hybrid_clock.cc:648] HybridClock initialized: now 1786063194739474 us; error 50 us; skew 500 ppm
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:54.742374 2882 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:39:54.742967 2882 webserver.cc:460] Webserver started at http://127.1.116.131:42821/ using document root <none> and password file <none>
I20260807 00:39:54.743249 2882 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:54.743324 2882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:54.743485 2882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:54.744704 2882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "801c4996b1324a228ebe741fb58c01f7"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "2da57f13ca7767da5048d50234f39466"
server_key_iv: "2f043c6bd130fa3fb95f93b9a69822a4"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.745126 2882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "801c4996b1324a228ebe741fb58c01f7"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "2da57f13ca7767da5048d50234f39466"
server_key_iv: "2f043c6bd130fa3fb95f93b9a69822a4"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.746797 2882 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:39:54.747781 2903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.748086 2882 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:54.748240 2882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "801c4996b1324a228ebe741fb58c01f7"
format_stamp: "Formatted at 2026-08-07 00:39:54 on dist-test-slave-h6n0"
server_key: "2da57f13ca7767da5048d50234f39466"
server_key_iv: "2f043c6bd130fa3fb95f93b9a69822a4"
server_key_version: "encryptionkey@0"
I20260807 00:39:54.748330 2882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:54.800251 2882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:54.802670 2882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:54.802855 2882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:54.803145 2882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:39:54.803524 2882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:54.803593 2882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.803632 2882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:54.803664 2882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:54.810969 2882 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:34973
I20260807 00:39:54.811051 3074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:34973 every 8 connection(s)
I20260807 00:39:54.811493 2882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:54.818734 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2882
I20260807 00:39:54.818832 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:39:54.819038 1490 external_mini_cluster.cc:1428] Setting key 078f5539e05d4df07a62ff281ed9be4c
I20260807 00:39:54.821856 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.812376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:57811 (local address 127.1.116.190:42651)
0807 00:39:54.812449 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.812451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.812855 (+ 404us) server_negotiation.cc:407] Connection header received
0807 00:39:54.813907 (+ 1052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.813908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.813922 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.813972 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.814667 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.815190 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.816212 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.816424 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.816520 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.818920 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.818921 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.818921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.818928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.820497 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.820990 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.820991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.820991 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.821041 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.821293 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.821294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.821294 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.821389 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.821444 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.821735 (+ 291us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.821793 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:39:54.822124 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.812361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42651 (local address 127.1.116.131:57811)
0807 00:39:54.812691 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.812753 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.813778 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.814046 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.814050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.814316 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:54.814520 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.814524 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.815287 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.815289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.816118 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.816121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.816533 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.816534 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.816648 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.817194 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:54.817203 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:54.818806 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:54.820611 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:54.820619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:54.820621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:54.820906 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:54.821113 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:54.821114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:54.821114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:54.821226 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:54.821454 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:54.821459 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:54.821595 (+ 136us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.821704 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.821852 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":221,"threads_started":1}
I20260807 00:39:54.822944 3075 heartbeater.cc:347] Connected to a master server at 127.1.116.190:42651
I20260807 00:39:54.823100 3075 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:54.823318 3075 heartbeater.cc:511] Master 127.1.116.190:42651 requested a full tablet report, sending...
I20260807 00:39:54.823735 2367 ts_manager.cc:194] Registered new tserver with Master: 801c4996b1324a228ebe741fb58c01f7 (127.1.116.131:34973)
I20260807 00:39:54.824221 2367 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:57811
I20260807 00:39:54.831221 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:39:54.839608 2433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.833212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46414 (local address 127.1.116.190:42651)
0807 00:39:54.833302 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.833309 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.833393 (+ 84us) server_negotiation.cc:407] Connection header received
0807 00:39:54.833499 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.833500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.833511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.833563 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.834172 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.834777 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.835572 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.835771 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.835850 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.836996 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.836998 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.836998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.837006 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.838247 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.838658 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.838659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.838660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.838745 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.838961 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.838962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.838962 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.839128 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.839290 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.839407 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.839491 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:39:54.842306 2367 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46414:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:39:54.844475 2367 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:39:54.852344 3093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.847867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:34973 (local address 127.0.0.1:55856)
0807 00:39:54.848663 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.848674 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.848736 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.848997 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.848998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.849001 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:54.849073 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.849076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.849913 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.849914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.851153 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.851157 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.851999 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.852000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.852087 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.852103 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.852180 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.852264 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":185,"threads_started":1}
I20260807 00:39:54.852371 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.848233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55856 (local address 127.1.116.131:34973)
0807 00:39:54.848472 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.848476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.848766 (+ 290us) server_negotiation.cc:407] Connection header received
0807 00:39:54.848813 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.848815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.848830 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.848910 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:54.849137 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.849799 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.851254 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.851901 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.851999 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.852067 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.852248 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.852296 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":149,"threads_started":1}
I20260807 00:39:54.852671 3094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.847788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44257 (local address 127.0.0.1:43546)
0807 00:39:54.847976 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.848021 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.848121 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.848597 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.848601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.848603 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:54.848696 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.848698 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.850161 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.850162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.851461 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.851464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.852404 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.852406 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.852482 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.852488 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.852524 (+ 36us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.852578 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":74,"thread_start_us":63,"threads_started":1}
I20260807 00:39:54.853073 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.848001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43546 (local address 127.1.116.129:44257)
0807 00:39:54.848309 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.848315 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.848353 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:39:54.848397 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.848399 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.848416 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.848576 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:54.848814 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.850018 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.851564 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.852305 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.852859 (+ 554us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.852921 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.852947 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.852989 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":130,"threads_started":1}
I20260807 00:39:54.853302 2965 tablet_service.cc:1467] Processing CreateTablet for tablet e3b7434d35b343a4a3484693c775fdf1 (DEFAULT_TABLE table=test-table [id=a4b98de97e7644d08c099d544a62bea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:54.853721 2965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b7434d35b343a4a3484693c775fdf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:54.853794 2566 tablet_service.cc:1467] Processing CreateTablet for tablet e3b7434d35b343a4a3484693c775fdf1 (DEFAULT_TABLE table=test-table [id=a4b98de97e7644d08c099d544a62bea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:54.854190 2566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b7434d35b343a4a3484693c775fdf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:54.856564 3100 tablet_bootstrap.cc:492] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7: Bootstrap starting.
I20260807 00:39:54.857079 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.848110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60660 (local address 127.1.116.130:41169)
0807 00:39:54.848343 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.848347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.848656 (+ 309us) server_negotiation.cc:407] Connection header received
0807 00:39:54.848708 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.848710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.848747 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.848839 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:54.849041 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.850113 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.851443 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.852234 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.852314 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.852378 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.856940 (+ 4562us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.856996 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":107,"threads_started":1}
I20260807 00:39:54.857360 3095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.847788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:41169 (local address 127.0.0.1:60660)
0807 00:39:54.848505 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.848519 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.848589 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.848891 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.848891 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.848893 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:54.848945 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.848946 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.850235 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.850236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.851338 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.851341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.852722 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.852724 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.856830 (+ 4106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.856836 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.857212 (+ 376us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.857275 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":688,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260807 00:39:54.857549 3100 tablet_bootstrap.cc:654] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:54.858018 3101 tablet_bootstrap.cc:492] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0: Bootstrap starting.
I20260807 00:39:54.858124 2766 tablet_service.cc:1467] Processing CreateTablet for tablet e3b7434d35b343a4a3484693c775fdf1 (DEFAULT_TABLE table=test-table [id=a4b98de97e7644d08c099d544a62bea3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:54.858444 3100 log.cc:824] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:54.858603 2766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e3b7434d35b343a4a3484693c775fdf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:54.858852 3101 tablet_bootstrap.cc:654] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:54.859656 3100 tablet_bootstrap.cc:492] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7: No bootstrap required, opened a new log
I20260807 00:39:54.859707 3101 log.cc:824] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:54.859834 3100 ts_tablet_manager.cc:1397] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:39:54.861301 3100 raft_consensus.cc:348] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.861442 3100 raft_consensus.cc:374] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:54.861464 3100 raft_consensus.cc:729] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 801c4996b1324a228ebe741fb58c01f7, State: Initialized, Role: FOLLOWER
I20260807 00:39:54.861586 3100 consensus_queue.cc:260] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.861893 3100 ts_tablet_manager.cc:1428] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:39:54.861938 3106 tablet_bootstrap.cc:492] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61: Bootstrap starting.
I20260807 00:39:54.862022 3075 heartbeater.cc:503] Master 127.1.116.190:42651 was elected leader, sending a full tablet report...
I20260807 00:39:54.862818 3106 tablet_bootstrap.cc:654] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:54.863138 3101 tablet_bootstrap.cc:492] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0: No bootstrap required, opened a new log
I20260807 00:39:54.863227 3101 ts_tablet_manager.cc:1397] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 00:39:54.863646 3106 log.cc:824] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:54.864496 3106 tablet_bootstrap.cc:492] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61: No bootstrap required, opened a new log
I20260807 00:39:54.864563 3106 ts_tablet_manager.cc:1397] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 00:39:54.865499 3101 raft_consensus.cc:348] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.865732 3101 raft_consensus.cc:374] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:54.865788 3101 raft_consensus.cc:729] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdbdf4a6e74c4d21b28aa6a60b0f8bf0, State: Initialized, Role: FOLLOWER
I20260807 00:39:54.865921 3101 consensus_queue.cc:260] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.866349 2670 heartbeater.cc:503] Master 127.1.116.190:42651 was elected leader, sending a full tablet report...
I20260807 00:39:54.866349 3101 ts_tablet_manager.cc:1428] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:39:54.866864 3106 raft_consensus.cc:348] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.867038 3106 raft_consensus.cc:374] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:54.867076 3106 raft_consensus.cc:729] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f441263682354fc08cb498decb3dcc61, State: Initialized, Role: FOLLOWER
I20260807 00:39:54.867204 3106 consensus_queue.cc:260] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.867585 2875 heartbeater.cc:503] Master 127.1.116.190:42651 was elected leader, sending a full tablet report...
I20260807 00:39:54.867619 3106 ts_tablet_manager.cc:1428] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260807 00:39:54.887923 3110 raft_consensus.cc:482] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:39:54.888016 3110 raft_consensus.cc:504] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.889087 3110 leader_election.cc:288] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdbdf4a6e74c4d21b28aa6a60b0f8bf0 (127.1.116.129:44257), 801c4996b1324a228ebe741fb58c01f7 (127.1.116.131:34973)
I20260807 00:39:54.891275 3107 raft_consensus.cc:482] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:39:54.891386 3107 raft_consensus.cc:504] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.891762 3107 leader_election.cc:288] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f441263682354fc08cb498decb3dcc61 (127.1.116.130:41169), bdbdf4a6e74c4d21b28aa6a60b0f8bf0 (127.1.116.129:44257)
I20260807 00:39:54.892647 2878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.889001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44257 (local address 127.1.116.130:44345)
0807 00:39:54.889091 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.889108 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.889176 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.889383 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.889384 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.889387 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:54.889464 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.889466 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.890352 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.890353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.891443 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.891446 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.892334 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.892335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.892424 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.892428 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.892478 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.892530 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 00:39:54.892670 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.889125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:44345 (local address 127.1.116.129:44257)
0807 00:39:54.889180 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.889182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.889192 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:39:54.889268 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.889269 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.889283 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.889345 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:54.889575 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.890251 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.891568 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.892149 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.892238 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.892249 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.892519 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.892596 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 00:39:54.893086 2598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b7434d35b343a4a3484693c775fdf1" candidate_uuid: "f441263682354fc08cb498decb3dcc61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" is_pre_election: true
I20260807 00:39:54.893304 2598 raft_consensus.cc:2454] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f441263682354fc08cb498decb3dcc61 in term 0.
I20260807 00:39:54.893599 2702 leader_election.cc:302] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdbdf4a6e74c4d21b28aa6a60b0f8bf0, f441263682354fc08cb498decb3dcc61; no voters:
I20260807 00:39:54.893684 3098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.889383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:52533 (local address 127.1.116.131:34973)
0807 00:39:54.889441 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.889444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.889687 (+ 243us) server_negotiation.cc:407] Connection header received
0807 00:39:54.889722 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.889723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.889733 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.889792 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:54.890006 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.890704 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.892642 (+ 1938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.893249 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.893297 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.893308 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.893567 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.893622 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:39:54.893968 3111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.889224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:34973 (local address 127.1.116.130:52533)
0807 00:39:54.889571 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.889583 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.889662 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.889807 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.889808 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.889810 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:54.889904 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.889906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.890819 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.890820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.892543 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.892546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.893399 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.893400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.893478 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.893483 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.893815 (+ 332us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.893878 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":1,"thread_start_us":182,"threads_started":1}
I20260807 00:39:54.894275 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b7434d35b343a4a3484693c775fdf1" candidate_uuid: "f441263682354fc08cb498decb3dcc61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801c4996b1324a228ebe741fb58c01f7" is_pre_election: true
I20260807 00:39:54.894398 2998 raft_consensus.cc:2454] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f441263682354fc08cb498decb3dcc61 in term 0.
I20260807 00:39:54.894344 3110 raft_consensus.cc:2781] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:39:54.894554 3110 raft_consensus.cc:482] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:39:54.894636 3110 raft_consensus.cc:3037] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:54.895450 3080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.891872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:41169 (local address 127.1.116.131:49869)
0807 00:39:54.891932 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.891944 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.891986 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.892107 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.892108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.892109 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:54.892173 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.892174 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.893258 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.893259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.894392 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.894395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.895191 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.895192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.895260 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.895263 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.895335 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.895382 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 00:39:54.895485 3097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.891936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:49869 (local address 127.1.116.130:41169)
0807 00:39:54.891999 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.892000 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.892011 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:39:54.892036 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.892037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.892048 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.892106 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:54.892441 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.893101 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.894504 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.895078 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.895149 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.895159 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.895364 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.895414 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":22}
I20260807 00:39:54.895890 2798 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b7434d35b343a4a3484693c775fdf1" candidate_uuid: "801c4996b1324a228ebe741fb58c01f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f441263682354fc08cb498decb3dcc61" is_pre_election: true
I20260807 00:39:54.896008 3110 raft_consensus.cc:504] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.896214 2798 raft_consensus.cc:2379] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 801c4996b1324a228ebe741fb58c01f7 in current term 1: Already voted for candidate f441263682354fc08cb498decb3dcc61 in this term.
I20260807 00:39:54.896376 2998 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b7434d35b343a4a3484693c775fdf1" candidate_uuid: "f441263682354fc08cb498decb3dcc61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "801c4996b1324a228ebe741fb58c01f7"
I20260807 00:39:54.896533 3110 leader_election.cc:288] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [CANDIDATE]: Term 1 election: Requested vote from peers bdbdf4a6e74c4d21b28aa6a60b0f8bf0 (127.1.116.129:44257), 801c4996b1324a228ebe741fb58c01f7 (127.1.116.131:34973)
I20260807 00:39:54.896498 2598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b7434d35b343a4a3484693c775fdf1" candidate_uuid: "f441263682354fc08cb498decb3dcc61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0"
I20260807 00:39:54.896514 2998 raft_consensus.cc:3037] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:54.896579 2598 raft_consensus.cc:3037] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:54.897600 2598 raft_consensus.cc:2454] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f441263682354fc08cb498decb3dcc61 in term 1.
I20260807 00:39:54.897789 2998 raft_consensus.cc:2454] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f441263682354fc08cb498decb3dcc61 in term 1.
I20260807 00:39:54.897884 2702 leader_election.cc:302] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bdbdf4a6e74c4d21b28aa6a60b0f8bf0, f441263682354fc08cb498decb3dcc61; no voters:
I20260807 00:39:54.898077 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.892279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:60331 (local address 127.1.116.129:44257)
0807 00:39:54.892766 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.892767 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.892775 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:39:54.892797 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.892798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.892807 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.892870 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:54.893075 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.893787 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.895157 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.895698 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.895795 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.895802 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.897937 (+ 2135us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.898006 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":130,"threads_started":1}
I20260807 00:39:54.898072 3110 raft_consensus.cc:2781] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:54.896503 3112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.892012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44257 (local address 127.1.116.131:60331)
0807 00:39:54.892274 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:54.892286 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:54.892343 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:54.892920 (+ 577us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:54.892921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:54.892923 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:54.892999 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.893001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.893893 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.893894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.895039 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:54.895042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.896172 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.896173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:54.896291 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.896296 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:39:54.896359 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 00:39:54.896403 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":145,"threads_started":1}
I20260807 00:39:54.898312 3110 raft_consensus.cc:686] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [term 1 LEADER]: Becoming Leader. State: Replica: f441263682354fc08cb498decb3dcc61, State: Running, Role: LEADER
I20260807 00:39:54.898419 2598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e3b7434d35b343a4a3484693c775fdf1" candidate_uuid: "801c4996b1324a228ebe741fb58c01f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" is_pre_election: true
I20260807 00:39:54.898424 3110 consensus_queue.cc:237] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } }
I20260807 00:39:54.898502 2598 raft_consensus.cc:2379] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 801c4996b1324a228ebe741fb58c01f7 in current term 1: Already voted for candidate f441263682354fc08cb498decb3dcc61 in this term.
I20260807 00:39:54.898839 2906 leader_election.cc:302] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 801c4996b1324a228ebe741fb58c01f7; no voters: bdbdf4a6e74c4d21b28aa6a60b0f8bf0, f441263682354fc08cb498decb3dcc61
I20260807 00:39:54.899026 3107 raft_consensus.cc:2726] T e3b7434d35b343a4a3484693c775fdf1 P 801c4996b1324a228ebe741fb58c01f7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 00:39:54.899469 2367 catalog_manager.cc:5495] T e3b7434d35b343a4a3484693c775fdf1 P f441263682354fc08cb498decb3dcc61 reported cstate change: term changed from 0 to 1, leader changed from <none> to f441263682354fc08cb498decb3dcc61 (127.1.116.130). New cstate: current_term: 1 leader_uuid: "f441263682354fc08cb498decb3dcc61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f441263682354fc08cb498decb3dcc61" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41169 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bdbdf4a6e74c4d21b28aa6a60b0f8bf0" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "801c4996b1324a228ebe741fb58c01f7" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 34973 } health_report { overall_health: UNKNOWN } } }
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:39:54.919203 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.910165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43554 (local address 127.1.116.129:44257)
0807 00:39:54.910229 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.910231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.910375 (+ 144us) server_negotiation.cc:407] Connection header received
0807 00:39:54.910456 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.910457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.910468 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.910555 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.911234 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.911677 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.912493 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.912688 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.912764 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.915736 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.915745 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.915745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.915753 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.917684 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.918152 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.918155 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.918155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.918228 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.918511 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.918512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.918512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.918744 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.918920 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.919070 (+ 150us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.919141 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260807 00:39:54.940891 2671 tablet.cc:2369] T e3b7434d35b343a4a3484693c775fdf1 P bdbdf4a6e74c4d21b28aa6a60b0f8bf0: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:39:54.949298 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.940757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43566 (local address 127.1.116.129:44257)
0807 00:39:54.940834 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.940838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.941030 (+ 192us) server_negotiation.cc:407] Connection header received
0807 00:39:54.941079 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.941080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.941093 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.941157 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.941785 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.942419 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.943695 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.943878 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.943950 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.946608 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.946610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.946610 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.946616 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.948059 (+ 1443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.948471 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.948472 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.948472 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.948527 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.948765 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.948766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.948766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.948867 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.948948 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.949174 (+ 226us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.949241 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:39:54 dist-test-slave-h6n0 krb5kdc[2293](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:39:54.976263 3096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:54.968894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43572 (local address 127.1.116.129:44257)
0807 00:39:54.968978 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:54.968982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:54.969091 (+ 109us) server_negotiation.cc:407] Connection header received
0807 00:39:54.969195 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:54.969196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:54.969208 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:54.969281 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:54.969856 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.970364 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.971211 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:54.971405 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:54.971483 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:54.973737 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:54.973738 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:54.973738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:54.973745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:54.975157 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.975516 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.975517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.975518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.975567 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:54.975789 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:54.975791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:54.975791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:54.975908 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:54.976010 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:54.976127 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 00:39:54.976193 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 00:39:54.976882 2540 tablet_service.cc:2908] Rejecting scan request for tablet e3b7434d35b343a4a3484693c775fdf1: Uninitialized: clean time has not yet been initialized
I20260807 00:39:54.978051 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2476
I20260807 00:39:54.989641 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2677
I20260807 00:39:55.000986 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2882
I20260807 00:39:55.014117 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2316
2026-08-07T00:39:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2291 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:39:55 dist-test-slave-h6n0 krb5kdc[3139](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:39:55 dist-test-slave-h6n0 krb5kdc[3139](info): set up 2 sockets
Aug 07 00:39:55 dist-test-slave-h6n0 krb5kdc[3139](info): commencing operation
krb5kdc: starting...
W20260807 00:39:56.262781 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:39:56 dist-test-slave-h6n0 krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:39:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:39:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:39:56.397651 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:40589
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:40589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:56.488863 3158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:56.489209 3158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:56.489253 3158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:56.490482 3158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:39:56.490573 3158 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 00:39:56.490597 3158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:56.490619 3158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:56.490653 3158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:39:56.490684 3158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:39:56.492255 3158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:40589
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:40589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3158
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:56.492558 3158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:56.492856 3158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:56.498397 3165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:56.498415 3166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:56.498521 3158 server_base.cc:1034] running on GCE node
W20260807 00:39:56.498415 3168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:56.499006 3158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:56.499399 3158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:56.500560 3158 hybrid_clock.cc:648] HybridClock initialized: now 1786063196500535 us; error 50 us; skew 500 ppm
Aug 07 00:39:56 dist-test-slave-h6n0 krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:56.503548 3158 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:39:56.504072 3158 webserver.cc:460] Webserver started at http://127.1.116.190:40323/ using document root <none> and password file <none>
I20260807 00:39:56.504361 3158 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:56.504415 3158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:56.504572 3158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:56.505784 3158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "f9220eb3d5ed4d468e4b8b933f25adf8"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "718f2e2618c3d7266c4417245b44a13d"
server_key_iv: "b3f427aefc2f7c260bf3376d480a100d"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.506177 3158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "f9220eb3d5ed4d468e4b8b933f25adf8"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "718f2e2618c3d7266c4417245b44a13d"
server_key_iv: "b3f427aefc2f7c260bf3376d480a100d"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.507784 3158 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:39:56.508791 3183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:56.509097 3158 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:39:56.509223 3158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "f9220eb3d5ed4d468e4b8b933f25adf8"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "718f2e2618c3d7266c4417245b44a13d"
server_key_iv: "b3f427aefc2f7c260bf3376d480a100d"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.509317 3158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:56.528918 3158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:56.531411 3158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:56.531611 3158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:56.538230 3158 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:40589
I20260807 00:39:56.538290 3273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:40589 every 8 connection(s)
I20260807 00:39:56.538719 3158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:39:56.539499 3274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:56.542013 3274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8: Bootstrap starting.
I20260807 00:39:56.542696 3274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:56.543025 3274 log.cc:824] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:56.543370 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 3158
I20260807 00:39:56.543460 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:39:56.543638 1490 external_mini_cluster.cc:1428] Setting key 5ba5040c32e9fd0c466e3d0e716e8b17
I20260807 00:39:56.544004 3274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8: No bootstrap required, opened a new log
I20260807 00:39:56.546514 3274 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40589 } }
I20260807 00:39:56.546743 3274 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:56.546813 3274 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9220eb3d5ed4d468e4b8b933f25adf8, State: Initialized, Role: FOLLOWER
I20260807 00:39:56.546988 3274 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40589 } }
I20260807 00:39:56.547097 3274 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:39:56.547133 3274 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:39:56.547183 3274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:56.547851 3274 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40589 } }
I20260807 00:39:56.547995 3274 leader_election.cc:302] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9220eb3d5ed4d468e4b8b933f25adf8; no voters:
I20260807 00:39:56.548210 3274 leader_election.cc:288] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:39:56.548300 3280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:56.548501 3274 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:39:56.548542 3280 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [term 1 LEADER]: Becoming Leader. State: Replica: f9220eb3d5ed4d468e4b8b933f25adf8, State: Running, Role: LEADER
I20260807 00:39:56.548727 3280 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40589 } }
I20260807 00:39:56.549197 3281 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40589 } } }
I20260807 00:39:56.549432 3281 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:56.549687 3282 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9220eb3d5ed4d468e4b8b933f25adf8. Latest consensus state: current_term: 1 leader_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9220eb3d5ed4d468e4b8b933f25adf8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40589 } } }
I20260807 00:39:56.549772 3282 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:56.549892 3288 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:39:56.550514 3288 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:39:56.552548 3288 catalog_manager.cc:1347] Generated new cluster ID: 34e131d520424a28a12af8d975894dba
I20260807 00:39:56.552613 3288 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:39:56 dist-test-slave-h6n0 krb5kdc[3139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:56.558318 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.544217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.1.116.190:40589)
0807 00:39:56.544407 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:56.544416 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:56.544481 (+ 65us) server_negotiation.cc:407] Connection header received
0807 00:39:56.546516 (+ 2035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:56.546556 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:56.546622 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:56.546799 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:56.547636 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.549142 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.550340 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.551081 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.551225 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.554255 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:56.554260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:56.554261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:56.554279 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:56.556513 (+ 2234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.556944 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.556947 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.556948 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.557069 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.557321 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.557322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.557322 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.557628 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:56.557786 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:56.557948 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 00:39:56.558107 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":90,"threads_started":1}
I20260807 00:39:56.567677 3288 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:39:56.568485 3288 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:39:56.574121 3288 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8: Generated new TSK 0
I20260807 00:39:56.574283 3288 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:39:56.609261 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40589
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:56.702016 3310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:56.702266 3310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:56.702292 3310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:56.703572 3310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:56.703619 3310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:56.703640 3310 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:39:56.703685 3310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:39:56.705255 3310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:56.705550 3310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:56.705827 3310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:56.711273 3324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:56.711266 3319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:56.711266 3321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:56.712003 3310 server_base.cc:1034] running on GCE node
I20260807 00:39:56.712322 3310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:56.712641 3310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:56.713817 3310 hybrid_clock.cc:648] HybridClock initialized: now 1786063196713804 us; error 50 us; skew 500 ppm
Aug 07 00:39:56 dist-test-slave-h6n0 krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:56.716743 3310 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:39:56.717275 3310 webserver.cc:460] Webserver started at http://127.1.116.129:45083/ using document root <none> and password file <none>
I20260807 00:39:56.717543 3310 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:56.717614 3310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:56.717777 3310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:56.719046 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "1c818a15c8d547df8051e0eb4d5ce048"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "9f5fdbef679073369b660395cbbbb541"
server_key_iv: "7ba77626622d3fc21088d3f7b9f29ff0"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.719453 3310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "1c818a15c8d547df8051e0eb4d5ce048"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "9f5fdbef679073369b660395cbbbb541"
server_key_iv: "7ba77626622d3fc21088d3f7b9f29ff0"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.720985 3310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:39:56.721895 3331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:56.722186 3310 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:56.722312 3310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "1c818a15c8d547df8051e0eb4d5ce048"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "9f5fdbef679073369b660395cbbbb541"
server_key_iv: "7ba77626622d3fc21088d3f7b9f29ff0"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.722394 3310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:56.739517 3310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:56.742054 3310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:56.742236 3310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:56.743039 3310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:56.743098 3310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:56.743139 3310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:56.743160 3310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:39:56 dist-test-slave-h6n0 krb5kdc[3139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:56.751960 3310 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:35509
I20260807 00:39:56.752068 3489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:35509 every 8 connection(s)
I20260807 00:39:56.752432 3310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:39:56.753752 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.743518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:42969 (local address 127.1.116.190:40589)
0807 00:39:56.743578 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:56.743581 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:56.743846 (+ 265us) server_negotiation.cc:407] Connection header received
0807 00:39:56.744450 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:56.744451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:56.744468 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:56.744526 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:56.745285 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.745832 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.746949 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.747210 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.747295 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.750456 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:56.750457 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:56.750458 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:56.750469 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:56.752232 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.752718 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.752719 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.752720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.752784 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.753071 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.753072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.753073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.753195 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:56.753277 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:56.753567 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 00:39:56.753656 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:39:56.753952 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.743516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40589 (local address 127.1.116.129:42969)
0807 00:39:56.743706 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:56.743750 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:56.744331 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:56.744607 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:56.744612 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:56.744897 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:56.745188 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.745193 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.745935 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.745936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.746852 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.746855 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.747305 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.747306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.747456 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.748057 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:56.748083 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:56.750357 (+ 2274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:56.752333 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.752339 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.752341 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.752632 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.752864 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.752865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.752865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.752998 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.753268 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:56.753276 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:56.753416 (+ 140us) client_negotiation.cc:769] Sending connection context
0807 00:39:56.753540 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 00:39:56.753703 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":108,"threads_started":1}
I20260807 00:39:56.754621 3490 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40589
I20260807 00:39:56.754719 3490 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:56.754935 3490 heartbeater.cc:511] Master 127.1.116.190:40589 requested a full tablet report, sending...
I20260807 00:39:56.755415 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 3310
I20260807 00:39:56.755532 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:39:56.755760 1490 external_mini_cluster.cc:1428] Setting key b575f1c54dba591cb14c29bfe1919f6b
I20260807 00:39:56.756245 3204 ts_manager.cc:194] Registered new tserver with Master: 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509)
I20260807 00:39:56.757001 3204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:42969
I20260807 00:39:56.763099 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.756031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:47443 (local address 127.1.116.190:40589)
0807 00:39:56.756069 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:56.756071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:56.756549 (+ 478us) server_negotiation.cc:407] Connection header received
0807 00:39:56.756609 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:56.756610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:56.756620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:56.756674 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:56.756993 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.757682 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.758867 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.759048 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.759110 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.760191 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:56.760192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:56.760192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:56.760200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:56.761743 (+ 1543us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.762224 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.762225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.762225 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.762273 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.762572 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.762572 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.762573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.762680 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:56.762737 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:56.762966 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 00:39:56.763047 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 00:39:56.763228 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.755873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40589 (local address 127.1.116.129:47443)
0807 00:39:56.756471 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:56.756480 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:56.756561 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:56.756679 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:56.756679 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:56.756866 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:56.756916 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.756918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.757814 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.757816 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.758766 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.758771 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.759138 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.759139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.759236 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.759771 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:56.759776 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:56.760089 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:56.761837 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.761838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.761838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.762135 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.762351 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.762352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.762352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.762448 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.762759 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:56.762761 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:56.762831 (+ 70us) client_negotiation.cc:769] Sending connection context
0807 00:39:56.762922 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 00:39:56.763072 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:39:56.785416 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40589
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:56.879351 3504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:56.879586 3504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:56.879611 3504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:56.880722 3504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:56.880759 3504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:56.880779 3504 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:39:56.880802 3504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:39:56.882289 3504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:56.882493 3504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:56.882725 3504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:56.888258 3512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:56.888348 3511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:56.888329 3516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:56.888811 3504 server_base.cc:1034] running on GCE node
I20260807 00:39:56.889056 3504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:56.889345 3504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:56.890501 3504 hybrid_clock.cc:648] HybridClock initialized: now 1786063196890463 us; error 53 us; skew 500 ppm
Aug 07 00:39:56 dist-test-slave-h6n0 krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:56.893441 3504 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:39:56.893970 3504 webserver.cc:460] Webserver started at http://127.1.116.130:37001/ using document root <none> and password file <none>
I20260807 00:39:56.894248 3504 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:56.894317 3504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:56.894493 3504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:56.895691 3504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "22bf67686fca458b9ccbecd0bc33861d"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "28c7eca1acafbb66e7c48231feb3ae9a"
server_key_iv: "6e9483ce59ea6bbc2201d1a3840bec57"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.896087 3504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "22bf67686fca458b9ccbecd0bc33861d"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "28c7eca1acafbb66e7c48231feb3ae9a"
server_key_iv: "6e9483ce59ea6bbc2201d1a3840bec57"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.897536 3504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:39:56.898343 3526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:56.898591 3504 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:56.898717 3504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "22bf67686fca458b9ccbecd0bc33861d"
format_stamp: "Formatted at 2026-08-07 00:39:56 on dist-test-slave-h6n0"
server_key: "28c7eca1acafbb66e7c48231feb3ae9a"
server_key_iv: "6e9483ce59ea6bbc2201d1a3840bec57"
server_key_version: "encryptionkey@0"
I20260807 00:39:56.898794 3504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:56.910909 3504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:56.913079 3504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:56.913247 3504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:56.913890 3504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:56.913949 3504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:56.913992 3504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:56.914022 3504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:39:56 dist-test-slave-h6n0 krb5kdc[3139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:56.923226 3504 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:39923
I20260807 00:39:56.923343 3691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:39923 every 8 connection(s)
I20260807 00:39:56.923727 3504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:39:56.924763 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.914390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:49849 (local address 127.1.116.190:40589)
0807 00:39:56.914483 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:56.914486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:56.914710 (+ 224us) server_negotiation.cc:407] Connection header received
0807 00:39:56.915249 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:56.915251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:56.915267 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:56.915370 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:56.916032 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.916583 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.917744 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.917999 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.918090 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.920897 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:56.920899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:56.920900 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:56.920910 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:56.922923 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.923550 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.923551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.923552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.923625 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.923994 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.923995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.923995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.924169 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:56.924245 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:56.924560 (+ 315us) server_negotiation.cc:299] Negotiation successful
0807 00:39:56.924661 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:39:56.924957 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.914362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40589 (local address 127.1.116.130:49849)
0807 00:39:56.914558 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:56.914600 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:56.915131 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:56.915429 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:56.915433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:56.915701 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:56.915920 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.915924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.916697 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.916698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.917657 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.917660 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.918108 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.918110 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.918237 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.918812 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:56.918826 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:56.920766 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:56.923062 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.923066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.923068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.923471 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.923704 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.923705 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.923705 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.923804 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.924255 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:56.924261 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:56.924408 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 00:39:56.924520 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 00:39:56.924707 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":91,"threads_started":1}
I20260807 00:39:56.925598 3693 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40589
I20260807 00:39:56.925689 3693 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:56.925907 3693 heartbeater.cc:511] Master 127.1.116.190:40589 requested a full tablet report, sending...
I20260807 00:39:56.926335 3204 ts_manager.cc:194] Registered new tserver with Master: 22bf67686fca458b9ccbecd0bc33861d (127.1.116.130:39923)
I20260807 00:39:56.926954 3204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:49849
I20260807 00:39:56.932186 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 3504
I20260807 00:39:56.932302 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:39:56.932492 1490 external_mini_cluster.cc:1428] Setting key 02edc68b8685914ccdeea81bd49984b0
I20260807 00:39:56.932958 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.926810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:51801 (local address 127.1.116.190:40589)
0807 00:39:56.926855 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:56.926856 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:56.926963 (+ 107us) server_negotiation.cc:407] Connection header received
0807 00:39:56.927059 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:56.927060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:56.927071 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:56.927139 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:56.927569 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.928121 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.928891 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.929068 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.929119 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.930000 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:56.930002 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:56.930002 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:56.930009 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:56.931540 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.931903 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.931904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.931904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.931951 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:56.932234 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:56.932236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:56.932236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:56.932467 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:56.932551 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:56.932744 (+ 193us) server_negotiation.cc:299] Negotiation successful
0807 00:39:56.932850 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 00:39:56.932976 3700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:56.926739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40589 (local address 127.1.116.130:51801)
0807 00:39:56.926892 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:56.926902 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:56.926968 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:56.927162 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:56.927163 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:56.927402 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:56.927481 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.927484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.928246 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.928247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.928810 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:56.928811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:56.929150 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:56.929151 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:56.929218 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:56.929618 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:56.929623 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:56.929915 (+ 292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:56.931642 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.931643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.931644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.931826 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.932059 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:56.932060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:56.932060 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:56.932125 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:56.932540 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:56.932541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:56.932593 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 00:39:56.932705 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 00:39:56.932844 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:39:56.962034 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40589
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35461
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:57.056290 3705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:57.056546 3705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:57.056572 3705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:57.057838 3705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:57.057885 3705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:57.057906 3705 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:39:57.057950 3705 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:39:57.059589 3705 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35461
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:40589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3705
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:57.059885 3705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:57.060171 3705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:57.066071 3714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:57.066100 3715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:57.066100 3717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:57.066295 3705 server_base.cc:1034] running on GCE node
I20260807 00:39:57.066695 3705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:57.067026 3705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:57.068215 3705 hybrid_clock.cc:648] HybridClock initialized: now 1786063197068185 us; error 48 us; skew 500 ppm
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063197, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:57.071115 3705 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:39:57.071707 3705 webserver.cc:460] Webserver started at http://127.1.116.131:36291/ using document root <none> and password file <none>
I20260807 00:39:57.071965 3705 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:57.072036 3705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:57.072206 3705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:57.073408 3705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "3a5d646362b84df9be83baaa97b0f601"
format_stamp: "Formatted at 2026-08-07 00:39:57 on dist-test-slave-h6n0"
server_key: "d52daf7d913430c7a04e2ef09fef1479"
server_key_iv: "a5acfd0d4ddc00785631049b2a04da82"
server_key_version: "encryptionkey@0"
I20260807 00:39:57.073825 3705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "3a5d646362b84df9be83baaa97b0f601"
format_stamp: "Formatted at 2026-08-07 00:39:57 on dist-test-slave-h6n0"
server_key: "d52daf7d913430c7a04e2ef09fef1479"
server_key_iv: "a5acfd0d4ddc00785631049b2a04da82"
server_key_version: "encryptionkey@0"
I20260807 00:39:57.075635 3705 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 00:39:57.076501 3727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:57.076731 3705 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:39:57.076849 3705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "3a5d646362b84df9be83baaa97b0f601"
format_stamp: "Formatted at 2026-08-07 00:39:57 on dist-test-slave-h6n0"
server_key: "d52daf7d913430c7a04e2ef09fef1479"
server_key_iv: "a5acfd0d4ddc00785631049b2a04da82"
server_key_version: "encryptionkey@0"
I20260807 00:39:57.076928 3705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:57.102178 3705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:57.104676 3705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:57.104866 3705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:57.105571 3705 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:57.105631 3705 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:57.105700 3705 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:57.105728 3705 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[3139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063197, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:57.114516 3705 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:38057
I20260807 00:39:57.114648 3905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:38057 every 8 connection(s)
I20260807 00:39:57.115145 3705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:39:57.115904 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.106000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:33149 (local address 127.1.116.190:40589)
0807 00:39:57.106086 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.106090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.106326 (+ 236us) server_negotiation.cc:407] Connection header received
0807 00:39:57.106928 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.106929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.106942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.107001 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:57.107755 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.108396 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.109495 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.109730 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.109811 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.112285 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:57.112286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:57.112287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:57.112294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:57.113975 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.114552 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.114555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.114555 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.114671 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.115052 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.115053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.115054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.115177 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:57.115303 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.115766 (+ 463us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.115839 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:39:57.116160 3740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.105991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40589 (local address 127.1.116.131:33149)
0807 00:39:57.106185 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.106227 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.106816 (+ 589us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.107083 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.107108 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.107448 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:57.107664 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.107668 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.108504 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.108505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.109389 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.109391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.109819 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.109820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.109942 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.110568 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:57.110578 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:57.112174 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:57.114085 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:57.114092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:57.114094 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:57.114404 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:57.114790 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:57.114791 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:57.114792 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:57.114941 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:57.115372 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:57.115378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:57.115587 (+ 209us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.115732 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.115906 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260807 00:39:57.116747 3906 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40589
I20260807 00:39:57.116863 3906 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:57.117067 3906 heartbeater.cc:511] Master 127.1.116.190:40589 requested a full tablet report, sending...
I20260807 00:39:57.117542 3208 ts_manager.cc:194] Registered new tserver with Master: 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131:38057)
I20260807 00:39:57.118103 3208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:33149
I20260807 00:39:57.118361 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 3705
I20260807 00:39:57.118448 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:39:57.118647 1490 external_mini_cluster.cc:1428] Setting key ff078557bb1e1aed8a6404dab5c53e53
I20260807 00:39:57.119691 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:39:57.124706 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.118033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:47597 (local address 127.1.116.190:40589)
0807 00:39:57.118140 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.118143 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.118178 (+ 35us) server_negotiation.cc:407] Connection header received
0807 00:39:57.118299 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.118300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.118319 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.118387 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:57.118921 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.119611 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.120442 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.120721 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.120805 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.121870 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:57.121871 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:57.121872 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:57.121880 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:57.123468 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.123843 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.123844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.123845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.123901 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.124157 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.124158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.124159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.124280 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:57.124356 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.124522 (+ 166us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.124613 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 00:39:57.124720 3915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.117888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40589 (local address 127.1.116.131:47597)
0807 00:39:57.118097 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.118111 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.118199 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.118399 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.118400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.118734 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:57.118825 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.118828 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.119741 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.119742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.120342 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.120344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.120833 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.120834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.120904 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.121450 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:57.121455 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:57.121761 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:57.123584 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:57.123585 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:57.123586 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:57.123764 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:57.123979 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:57.123980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:57.123980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:57.124041 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:57.124358 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:57.124359 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:57.124406 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.124495 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.124609 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":88,"threads_started":1}
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[3139](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063197, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[3139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063197, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:57.145040 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.137804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34470 (local address 127.1.116.190:40589)
0807 00:39:57.137901 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.137904 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.137916 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:39:57.137945 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.137947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.137957 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.138036 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:57.138635 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.139167 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.139972 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.140174 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.140238 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.142449 (+ 2211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:57.142450 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:57.142450 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:57.142456 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:57.144020 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.144359 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.144360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.144360 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.144412 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.144609 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.144610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.144610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.144712 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:57.144798 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.144902 (+ 104us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.144966 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:39:57.147153 3208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:39:57.149189 3208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:39:57.156949 3937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.152359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57474 (local address 127.1.116.130:39923)
0807 00:39:57.152723 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.152727 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.152735 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:39:57.152767 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.152768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.152780 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.152865 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:57.153077 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.153969 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.155579 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.156502 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.156596 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.156669 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.156765 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.156819 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":117,"threads_started":1}
I20260807 00:39:57.156937 3934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.152196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:38057 (local address 127.0.0.1:33310)
0807 00:39:57.152428 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.152442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.152504 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.152996 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.152997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.153000 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:57.153039 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.153040 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.154518 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.154519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.155677 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.155680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.156434 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.156435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.156558 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.156582 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.156663 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.156742 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":2,"thread_start_us":116,"threads_started":1}
I20260807 00:39:57.157012 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.152263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:39923 (local address 127.0.0.1:57474)
0807 00:39:57.152585 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.152602 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.152637 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.152892 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.152895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.152897 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:57.152979 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.152981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.154101 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.154102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.155455 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.155459 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.156601 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.156603 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.156670 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.156675 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.156734 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.156789 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":88,"threads_started":1}
I20260807 00:39:57.157418 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.152499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33310 (local address 127.1.116.131:38057)
0807 00:39:57.152765 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.152771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.152785 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:39:57.152861 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.152864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.152885 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.153004 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:57.153112 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.154417 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.155773 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.156348 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.156432 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.156488 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.157226 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.157297 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":183,"threads_started":1}
I20260807 00:39:57.157760 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.152173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35509 (local address 127.0.0.1:41322)
0807 00:39:57.152354 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.152393 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.152466 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.153470 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.153471 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.153473 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:57.153510 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.153511 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.154874 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.154876 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.156242 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.156246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.157462 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.157463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.157545 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.157549 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.157627 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.157682 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":3,"thread_start_us":72,"threads_started":1}
I20260807 00:39:57.157773 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.152821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.1.116.129:35509)
0807 00:39:57.153225 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.153229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.153241 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:39:57.153287 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.153289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.153306 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.153441 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:57.153593 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.154774 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.156368 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.157373 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.157448 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.157504 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.157644 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.157684 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":139,"threads_started":1}
I20260807 00:39:57.158120 3592 tablet_service.cc:1467] Processing CreateTablet for tablet e8c1b4c9c0914d01b62591ce04763879 (DEFAULT_TABLE table=test-table [id=35b3103c0c724819aa10269ad5d99d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:57.158245 3811 tablet_service.cc:1467] Processing CreateTablet for tablet e8c1b4c9c0914d01b62591ce04763879 (DEFAULT_TABLE table=test-table [id=35b3103c0c724819aa10269ad5d99d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:57.158634 3811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c1b4c9c0914d01b62591ce04763879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.158634 3592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c1b4c9c0914d01b62591ce04763879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.158505 3402 tablet_service.cc:1467] Processing CreateTablet for tablet e8c1b4c9c0914d01b62591ce04763879 (DEFAULT_TABLE table=test-table [id=35b3103c0c724819aa10269ad5d99d1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:57.158880 3402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8c1b4c9c0914d01b62591ce04763879. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.161574 3942 tablet_bootstrap.cc:492] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Bootstrap starting.
I20260807 00:39:57.162150 3942 tablet_bootstrap.cc:654] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.162386 3942 log.cc:824] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:57.162746 3943 tablet_bootstrap.cc:492] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Bootstrap starting.
I20260807 00:39:57.163164 3942 tablet_bootstrap.cc:492] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: No bootstrap required, opened a new log
I20260807 00:39:57.163245 3942 ts_tablet_manager.cc:1397] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 00:39:57.163389 3945 tablet_bootstrap.cc:492] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Bootstrap starting.
I20260807 00:39:57.163695 3943 tablet_bootstrap.cc:654] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.164018 3945 tablet_bootstrap.cc:654] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.164041 3943 log.cc:824] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:57.164402 3945 log.cc:824] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:57.164938 3943 tablet_bootstrap.cc:492] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: No bootstrap required, opened a new log
I20260807 00:39:57.165006 3943 ts_tablet_manager.cc:1397] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:39:57.165211 3945 tablet_bootstrap.cc:492] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: No bootstrap required, opened a new log
I20260807 00:39:57.165263 3945 ts_tablet_manager.cc:1397] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:39:57.165284 3942 raft_consensus.cc:348] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.165444 3942 raft_consensus.cc:374] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.165475 3942 raft_consensus.cc:729] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c818a15c8d547df8051e0eb4d5ce048, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.165604 3942 consensus_queue.cc:260] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.165868 3942 ts_tablet_manager.cc:1428] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:39:57.166440 3490 heartbeater.cc:503] Master 127.1.116.190:40589 was elected leader, sending a full tablet report...
I20260807 00:39:57.167145 3943 raft_consensus.cc:348] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.167338 3943 raft_consensus.cc:374] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.167424 3943 raft_consensus.cc:729] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5d646362b84df9be83baaa97b0f601, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.167567 3943 consensus_queue.cc:260] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.167678 3945 raft_consensus.cc:348] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.167858 3943 ts_tablet_manager.cc:1428] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:39:57.168002 3906 heartbeater.cc:503] Master 127.1.116.190:40589 was elected leader, sending a full tablet report...
I20260807 00:39:57.167948 3945 raft_consensus.cc:374] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.168381 3945 raft_consensus.cc:729] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22bf67686fca458b9ccbecd0bc33861d, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.168553 3945 consensus_queue.cc:260] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.168956 3945 ts_tablet_manager.cc:1428] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260807 00:39:57.169047 3693 heartbeater.cc:503] Master 127.1.116.190:40589 was elected leader, sending a full tablet report...
W20260807 00:39:57.174364 3694 tablet.cc:2369] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:39:57.229138 3952 raft_consensus.cc:482] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:39:57.229282 3952 raft_consensus.cc:504] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.229666 3952 leader_election.cc:288] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131:38057), 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509)
I20260807 00:39:57.233074 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.229797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:38057 (local address 127.1.116.130:42219)
0807 00:39:57.229850 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.229862 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.229904 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.230112 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.230113 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.230115 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:57.230193 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.230195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.231062 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.231064 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.232124 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.232127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.232776 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.232777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.232872 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.232877 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.232946 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.232992 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":3}
I20260807 00:39:57.233120 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.229850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:42219 (local address 127.1.116.131:38057)
0807 00:39:57.229925 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.229928 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.229964 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:39:57.230004 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.230006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.230021 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.230091 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:57.230295 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.230974 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.232209 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.232695 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.232754 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.232763 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.232961 (+ 198us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.233023 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 00:39:57.233417 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.230060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:39819 (local address 127.1.116.129:35509)
0807 00:39:57.230122 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.230124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.230268 (+ 144us) server_negotiation.cc:407] Connection header received
0807 00:39:57.230335 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.230336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.230344 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.230392 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:57.230618 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.231133 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.232500 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.232930 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.232998 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.233005 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.233266 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.233340 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 00:39:57.233456 3838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1b4c9c0914d01b62591ce04763879" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5d646362b84df9be83baaa97b0f601" is_pre_election: true
I20260807 00:39:57.233613 3838 raft_consensus.cc:2454] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 0.
I20260807 00:39:57.233788 3954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.229928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35509 (local address 127.1.116.130:39819)
0807 00:39:57.230176 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.230186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.230275 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.230420 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.230421 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.230424 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:57.230515 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.230517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.231225 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.231225 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.232405 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.232408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.233017 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.233018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.233103 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.233107 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.233529 (+ 422us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.233590 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":2,"thread_start_us":143,"threads_started":1}
I20260807 00:39:57.233883 3530 leader_election.cc:302] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22bf67686fca458b9ccbecd0bc33861d, 3a5d646362b84df9be83baaa97b0f601; no voters:
I20260807 00:39:57.234012 3952 raft_consensus.cc:2781] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:39:57.234069 3952 raft_consensus.cc:482] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:39:57.234098 3952 raft_consensus.cc:3037] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.234184 3431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1b4c9c0914d01b62591ce04763879" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c818a15c8d547df8051e0eb4d5ce048" is_pre_election: true
I20260807 00:39:57.234364 3431 raft_consensus.cc:2454] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 0.
I20260807 00:39:57.235402 3952 raft_consensus.cc:504] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.235570 3952 leader_election.cc:288] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 election: Requested vote from peers 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131:38057), 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509)
I20260807 00:39:57.235723 3838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1b4c9c0914d01b62591ce04763879" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5d646362b84df9be83baaa97b0f601"
I20260807 00:39:57.235795 3431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8c1b4c9c0914d01b62591ce04763879" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c818a15c8d547df8051e0eb4d5ce048"
I20260807 00:39:57.235816 3838 raft_consensus.cc:3037] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.235911 3431 raft_consensus.cc:3037] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.236687 3838 raft_consensus.cc:2454] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 1.
I20260807 00:39:57.236914 3530 leader_election.cc:302] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 22bf67686fca458b9ccbecd0bc33861d, 3a5d646362b84df9be83baaa97b0f601; no voters:
I20260807 00:39:57.237052 3952 raft_consensus.cc:2781] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:57.237089 3431 raft_consensus.cc:2454] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 1.
I20260807 00:39:57.237299 3952 raft_consensus.cc:686] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 1 LEADER]: Becoming Leader. State: Replica: 22bf67686fca458b9ccbecd0bc33861d, State: Running, Role: LEADER
I20260807 00:39:57.237484 3952 consensus_queue.cc:237] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } }
I20260807 00:39:57.238492 3208 catalog_manager.cc:5495] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d reported cstate change: term changed from 0 to 1, leader changed from <none> to 22bf67686fca458b9ccbecd0bc33861d (127.1.116.130). New cstate: current_term: 1 leader_uuid: "22bf67686fca458b9ccbecd0bc33861d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } health_report { overall_health: HEALTHY } } }
W20260807 00:39:57.253332 3492 tablet.cc:2369] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:39:57.257161 3937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.254467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.1.116.130:39923)
0807 00:39:57.254581 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.254583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.254595 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:39:57.254674 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.254675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.254693 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.254779 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:39:57.254927 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.255479 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.256354 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.256542 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.256602 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.256801 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:39:57.256902 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:39:57.256966 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.257055 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.257076 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 00:39:57.259480 3431 raft_consensus.cc:1261] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Refusing update from remote peer 22bf67686fca458b9ccbecd0bc33861d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:39:57.259460 3838 raft_consensus.cc:1261] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Refusing update from remote peer 22bf67686fca458b9ccbecd0bc33861d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:39:57.259908 3952 consensus_queue.cc:1035] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:39:57.260082 3956 consensus_queue.cc:1035] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:39:57.262814 3959 mvcc.cc:205] Tried to move back new op lower bound from 7315714855971176448 to 7315714855885537280. Current Snapshot: MvccSnapshot[applied={T|T < 7315714855971176448}]
I20260807 00:39:57.264396 3964 mvcc.cc:205] Tried to move back new op lower bound from 7315714855971176448 to 7315714855885537280. Current Snapshot: MvccSnapshot[applied={T|T < 7315714855971176448}]
I20260807 00:39:57.267676 3208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:39:57.267822 3208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:39:57.268833 3208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8: Sending DeleteTablet for 3 replicas of tablet e8c1b4c9c0914d01b62591ce04763879
I20260807 00:39:57.269405 3402 tablet_service.cc:1514] Processing DeleteTablet for tablet e8c1b4c9c0914d01b62591ce04763879 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:39:57 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:41322
I20260807 00:39:57.269438 3592 tablet_service.cc:1514] Processing DeleteTablet for tablet e8c1b4c9c0914d01b62591ce04763879 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:39:57 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:57474
I20260807 00:39:57.269407 3811 tablet_service.cc:1514] Processing DeleteTablet for tablet e8c1b4c9c0914d01b62591ce04763879 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:39:57 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:33310
I20260807 00:39:57.269737 3973 tablet_replica.cc:330] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: stopping tablet replica
I20260807 00:39:57.269793 3974 tablet_replica.cc:330] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: stopping tablet replica
I20260807 00:39:57.269907 3973 raft_consensus.cc:2229] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:39:57.269907 3974 raft_consensus.cc:2229] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:39:57.270043 3974 raft_consensus.cc:2258] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:39:57.270051 3973 raft_consensus.cc:2258] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:39:57.270025 3208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:39:57.270233 3208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:39:57.270243 3972 tablet_replica.cc:330] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: stopping tablet replica
I20260807 00:39:57.270792 3973 ts_tablet_manager.cc:1905] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:39:57.271061 3974 ts_tablet_manager.cc:1905] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:39:57.271938 3592 tablet_service.cc:1467] Processing CreateTablet for tablet 3659a986bc8342e6ac88415a946ca3fc (DEFAULT_TABLE table=test-table [id=3706ad20bb6a407c9cdcd73dbcda3b6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:57.272104 3592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3659a986bc8342e6ac88415a946ca3fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.273331 3973 ts_tablet_manager.cc:1918] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:39:57.273411 3973 log.cc:1196] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/e8c1b4c9c0914d01b62591ce04763879
I20260807 00:39:57.273455 3974 ts_tablet_manager.cc:1918] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:39:57.273520 3974 log.cc:1196] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/e8c1b4c9c0914d01b62591ce04763879
I20260807 00:39:57.273736 3973 ts_tablet_manager.cc:1939] T e8c1b4c9c0914d01b62591ce04763879 P 22bf67686fca458b9ccbecd0bc33861d: Deleting consensus metadata
I20260807 00:39:57.273844 3945 tablet_bootstrap.cc:492] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: Bootstrap starting.
I20260807 00:39:57.274219 3190 catalog_manager.cc:4841] TS 22bf67686fca458b9ccbecd0bc33861d (127.1.116.130:39923): tablet e8c1b4c9c0914d01b62591ce04763879 (table test-table [id=35b3103c0c724819aa10269ad5d99d1b]) successfully deleted
I20260807 00:39:57.274350 3945 tablet_bootstrap.cc:654] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.274339 3402 tablet_service.cc:1467] Processing CreateTablet for tablet 3659a986bc8342e6ac88415a946ca3fc (DEFAULT_TABLE table=test-table [id=3706ad20bb6a407c9cdcd73dbcda3b6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:57.274464 3402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3659a986bc8342e6ac88415a946ca3fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.270360 3972 raft_consensus.cc:2229] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:39:57.274751 3972 raft_consensus.cc:2258] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:39:57.274102 3811 tablet_service.cc:1467] Processing CreateTablet for tablet 3659a986bc8342e6ac88415a946ca3fc (DEFAULT_TABLE table=test-table [id=3706ad20bb6a407c9cdcd73dbcda3b6d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:57.275035 3811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3659a986bc8342e6ac88415a946ca3fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.275893 3942 tablet_bootstrap.cc:492] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: Bootstrap starting.
I20260807 00:39:57.275906 3943 tablet_bootstrap.cc:492] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Bootstrap starting.
I20260807 00:39:57.276352 3942 tablet_bootstrap.cc:654] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.276425 3943 tablet_bootstrap.cc:654] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.276446 3972 ts_tablet_manager.cc:1905] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:39:57.277034 3943 tablet_bootstrap.cc:492] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: No bootstrap required, opened a new log
I20260807 00:39:57.277123 3943 ts_tablet_manager.cc:1397] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:57.277307 3943 raft_consensus.cc:348] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.277407 3943 raft_consensus.cc:374] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.277429 3943 raft_consensus.cc:729] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5d646362b84df9be83baaa97b0f601, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.277477 3943 consensus_queue.cc:260] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.277585 3943 ts_tablet_manager.cc:1428] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:57.277922 3942 tablet_bootstrap.cc:492] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: No bootstrap required, opened a new log
I20260807 00:39:57.277968 3942 ts_tablet_manager.cc:1397] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 00:39:57.278203 3942 raft_consensus.cc:348] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.278294 3942 raft_consensus.cc:374] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.278318 3942 raft_consensus.cc:729] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c818a15c8d547df8051e0eb4d5ce048, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.278359 3942 consensus_queue.cc:260] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.278407 3972 ts_tablet_manager.cc:1918] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:39:57.273778 3974 ts_tablet_manager.cc:1939] T e8c1b4c9c0914d01b62591ce04763879 P 3a5d646362b84df9be83baaa97b0f601: Deleting consensus metadata
I20260807 00:39:57.278489 3942 ts_tablet_manager.cc:1428] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:39:57.278482 3972 log.cc:1196] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/e8c1b4c9c0914d01b62591ce04763879
I20260807 00:39:57.279114 3186 catalog_manager.cc:4841] TS 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131:38057): tablet e8c1b4c9c0914d01b62591ce04763879 (table test-table [id=35b3103c0c724819aa10269ad5d99d1b]) successfully deleted
I20260807 00:39:57.279472 3972 ts_tablet_manager.cc:1939] T e8c1b4c9c0914d01b62591ce04763879 P 1c818a15c8d547df8051e0eb4d5ce048: Deleting consensus metadata
I20260807 00:39:57.279932 3184 catalog_manager.cc:4841] TS 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509): tablet e8c1b4c9c0914d01b62591ce04763879 (table test-table [id=35b3103c0c724819aa10269ad5d99d1b]) successfully deleted
I20260807 00:39:57.280244 3945 tablet_bootstrap.cc:492] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: No bootstrap required, opened a new log
I20260807 00:39:57.280293 3945 ts_tablet_manager.cc:1397] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260807 00:39:57.280472 3945 raft_consensus.cc:348] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.280536 3945 raft_consensus.cc:374] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.280557 3945 raft_consensus.cc:729] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22bf67686fca458b9ccbecd0bc33861d, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.280609 3945 consensus_queue.cc:260] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.280700 3945 ts_tablet_manager.cc:1428] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
W20260807 00:39:57.365898 3907 tablet.cc:2369] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:39:57.457324 3956 raft_consensus.cc:482] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:39:57.457471 3956 raft_consensus.cc:504] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.457738 3956 leader_election.cc:288] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509), 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131:38057)
I20260807 00:39:57.458010 3431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3659a986bc8342e6ac88415a946ca3fc" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c818a15c8d547df8051e0eb4d5ce048" is_pre_election: true
I20260807 00:39:57.458153 3431 raft_consensus.cc:2454] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 0.
I20260807 00:39:57.458081 3838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3659a986bc8342e6ac88415a946ca3fc" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5d646362b84df9be83baaa97b0f601" is_pre_election: true
I20260807 00:39:57.458245 3838 raft_consensus.cc:2454] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 0.
I20260807 00:39:57.458374 3527 leader_election.cc:302] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c818a15c8d547df8051e0eb4d5ce048, 22bf67686fca458b9ccbecd0bc33861d; no voters:
I20260807 00:39:57.458583 3956 raft_consensus.cc:2781] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:39:57.458684 3956 raft_consensus.cc:482] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:39:57.458727 3956 raft_consensus.cc:3037] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.459658 3956 raft_consensus.cc:504] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.459848 3956 leader_election.cc:288] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 election: Requested vote from peers 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509), 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131:38057)
I20260807 00:39:57.460012 3838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3659a986bc8342e6ac88415a946ca3fc" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a5d646362b84df9be83baaa97b0f601"
I20260807 00:39:57.460062 3431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3659a986bc8342e6ac88415a946ca3fc" candidate_uuid: "22bf67686fca458b9ccbecd0bc33861d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c818a15c8d547df8051e0eb4d5ce048"
I20260807 00:39:57.460119 3838 raft_consensus.cc:3037] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.460137 3431 raft_consensus.cc:3037] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.461081 3838 raft_consensus.cc:2454] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 1.
I20260807 00:39:57.461089 3431 raft_consensus.cc:2454] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 22bf67686fca458b9ccbecd0bc33861d in term 1.
I20260807 00:39:57.461347 3527 leader_election.cc:302] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c818a15c8d547df8051e0eb4d5ce048, 22bf67686fca458b9ccbecd0bc33861d; no voters:
I20260807 00:39:57.461490 3956 raft_consensus.cc:2781] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:57.461602 3956 raft_consensus.cc:686] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 1 LEADER]: Becoming Leader. State: Replica: 22bf67686fca458b9ccbecd0bc33861d, State: Running, Role: LEADER
I20260807 00:39:57.461675 3956 consensus_queue.cc:237] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.462447 3208 catalog_manager.cc:5495] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d reported cstate change: term changed from 0 to 1, leader changed from <none> to 22bf67686fca458b9ccbecd0bc33861d (127.1.116.130). New cstate: current_term: 1 leader_uuid: "22bf67686fca458b9ccbecd0bc33861d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } health_report { overall_health: UNKNOWN } } }
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[3139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063196, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:57.547140 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.539540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34482 (local address 127.1.116.190:40589)
0807 00:39:57.539656 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.539660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.539800 (+ 140us) server_negotiation.cc:407] Connection header received
0807 00:39:57.539896 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.539897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.539906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.539969 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:57.540501 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.541029 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.541733 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.541916 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.541992 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.544405 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:57.544406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:57.544407 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:57.544414 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:57.545840 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.546286 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.546287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.546288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.546345 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.546571 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.546572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.546572 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.546697 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:57.546774 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.546937 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.547026 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 00:39:57.547236 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.539460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40589 (local address 127.0.0.1:34482)
0807 00:39:57.539693 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.539704 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.539822 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.539974 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.539975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.540348 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:57.540407 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.540409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.541115 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.541116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.541648 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.541649 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.541990 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.541991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.542044 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.542727 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:57.542732 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:57.544309 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:57.545943 (+ 1634us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:57.545951 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:57.545954 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:57.546193 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:57.546421 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:57.546422 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:57.546422 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:57.546484 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:57.546775 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:57.546783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:57.546828 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.546913 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.547108 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":81,"thread_start_us":124,"threads_started":1}
I20260807 00:39:57.549758 3208 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:34482:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 00:39:57.550019 3208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:39:57.552138 3811 tablet_service.cc:1467] Processing CreateTablet for tablet 0a4b243162cc4daab0f2dd98518ebf43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e81203b4cc84d6c9f478e7664af242f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:39:57.552358 3811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a4b243162cc4daab0f2dd98518ebf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.552126 3402 tablet_service.cc:1467] Processing CreateTablet for tablet 0a4b243162cc4daab0f2dd98518ebf43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e81203b4cc84d6c9f478e7664af242f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:39:57.553675 3402 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a4b243162cc4daab0f2dd98518ebf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.554513 3943 tablet_bootstrap.cc:492] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: Bootstrap starting.
I20260807 00:39:57.555039 3942 tablet_bootstrap.cc:492] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: Bootstrap starting.
I20260807 00:39:57.555162 3943 tablet_bootstrap.cc:654] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.555143 3592 tablet_service.cc:1467] Processing CreateTablet for tablet 0a4b243162cc4daab0f2dd98518ebf43 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5e81203b4cc84d6c9f478e7664af242f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:39:57.555305 3592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a4b243162cc4daab0f2dd98518ebf43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:57.555667 3942 tablet_bootstrap.cc:654] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.556455 3942 tablet_bootstrap.cc:492] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: No bootstrap required, opened a new log
I20260807 00:39:57.556535 3942 ts_tablet_manager.cc:1397] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 00:39:57.556803 3945 tablet_bootstrap.cc:492] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: Bootstrap starting.
I20260807 00:39:57.556759 3942 raft_consensus.cc:348] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.556855 3942 raft_consensus.cc:374] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.556876 3942 raft_consensus.cc:729] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c818a15c8d547df8051e0eb4d5ce048, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.556926 3942 consensus_queue.cc:260] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.557041 3942 ts_tablet_manager.cc:1428] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:39:57.557134 3943 tablet_bootstrap.cc:492] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: No bootstrap required, opened a new log
I20260807 00:39:57.557188 3943 ts_tablet_manager.cc:1397] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 00:39:57.557260 3945 tablet_bootstrap.cc:654] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:57.557235 3949 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.557344 3949 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:39:57.557351 3943 raft_consensus.cc:348] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.557423 3943 raft_consensus.cc:374] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.557442 3943 raft_consensus.cc:729] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a5d646362b84df9be83baaa97b0f601, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.557487 3943 consensus_queue.cc:260] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.557583 3943 ts_tablet_manager.cc:1428] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:39:57.557723 3950 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.557798 3950 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:39:57.566045 3945 tablet_bootstrap.cc:492] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: No bootstrap required, opened a new log
I20260807 00:39:57.566116 3945 ts_tablet_manager.cc:1397] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.001s
I20260807 00:39:57.566250 3945 raft_consensus.cc:348] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.566349 3945 raft_consensus.cc:374] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:57.566375 3945 raft_consensus.cc:729] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22bf67686fca458b9ccbecd0bc33861d, State: Initialized, Role: FOLLOWER
I20260807 00:39:57.566419 3945 consensus_queue.cc:260] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.566543 3945 ts_tablet_manager.cc:1428] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:39:57.566561 3956 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.566658 3956 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:39:57.679306 3950 raft_consensus.cc:482] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:39:57.679457 3950 raft_consensus.cc:504] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.680024 3950 leader_election.cc:288] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509), 22bf67686fca458b9ccbecd0bc33861d (127.1.116.130:39923)
I20260807 00:39:57.684742 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.680268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:58389 (local address 127.1.116.129:35509)
0807 00:39:57.680331 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.680334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.680410 (+ 76us) server_negotiation.cc:407] Connection header received
0807 00:39:57.680565 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.680566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.680579 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.680667 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:57.680972 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.681900 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.683654 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.684240 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.684336 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.684348 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.684588 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.684666 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 00:39:57.684742 3937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.680429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51733 (local address 127.1.116.130:39923)
0807 00:39:57.680512 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.680514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.680527 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:39:57.680565 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.680566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.680582 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.680657 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:57.680974 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.681902 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.683655 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.684243 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.684336 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.684348 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.684588 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.684657 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:39:57.684708 3995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.680178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35509 (local address 127.1.116.131:58389)
0807 00:39:57.680312 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.680325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.680453 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.680697 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.680698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.680702 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:57.680827 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.680831 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.682014 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.682015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.683517 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.683520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.684359 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.684360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.684450 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.684467 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.684529 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.684583 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":131,"thread_start_us":77,"threads_started":1}
I20260807 00:39:57.684708 3997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.680290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:39923 (local address 127.1.116.131:51733)
0807 00:39:57.680423 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.680434 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.680484 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.680697 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.680698 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.680700 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:57.680827 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.680830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.682012 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.682014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.683517 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.683520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.684357 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.684359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.684450 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.684459 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.684529 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.684583 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":92,"thread_start_us":81,"threads_started":1}
I20260807 00:39:57.685155 3431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4b243162cc4daab0f2dd98518ebf43" candidate_uuid: "3a5d646362b84df9be83baaa97b0f601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c818a15c8d547df8051e0eb4d5ce048" is_pre_election: true
I20260807 00:39:57.685364 3431 raft_consensus.cc:2454] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5d646362b84df9be83baaa97b0f601 in term 0.
I20260807 00:39:57.685617 3728 leader_election.cc:302] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c818a15c8d547df8051e0eb4d5ce048, 3a5d646362b84df9be83baaa97b0f601; no voters:
I20260807 00:39:57.685587 3624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4b243162cc4daab0f2dd98518ebf43" candidate_uuid: "3a5d646362b84df9be83baaa97b0f601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22bf67686fca458b9ccbecd0bc33861d" is_pre_election: true
I20260807 00:39:57.685755 3624 raft_consensus.cc:2454] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a5d646362b84df9be83baaa97b0f601 in term 0.
I20260807 00:39:57.685808 3950 raft_consensus.cc:2781] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:39:57.685854 3950 raft_consensus.cc:482] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:39:57.685896 3950 raft_consensus.cc:3037] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.687023 3950 raft_consensus.cc:504] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.687227 3950 leader_election.cc:288] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [CANDIDATE]: Term 1 election: Requested vote from peers 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509), 22bf67686fca458b9ccbecd0bc33861d (127.1.116.130:39923)
I20260807 00:39:57.687386 3431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4b243162cc4daab0f2dd98518ebf43" candidate_uuid: "3a5d646362b84df9be83baaa97b0f601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c818a15c8d547df8051e0eb4d5ce048"
I20260807 00:39:57.687470 3431 raft_consensus.cc:3037] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.687450 3624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a4b243162cc4daab0f2dd98518ebf43" candidate_uuid: "3a5d646362b84df9be83baaa97b0f601" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "22bf67686fca458b9ccbecd0bc33861d"
I20260807 00:39:57.687517 3624 raft_consensus.cc:3037] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:57.688338 3431 raft_consensus.cc:2454] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5d646362b84df9be83baaa97b0f601 in term 1.
I20260807 00:39:57.688536 3624 raft_consensus.cc:2454] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a5d646362b84df9be83baaa97b0f601 in term 1.
I20260807 00:39:57.688575 3728 leader_election.cc:302] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c818a15c8d547df8051e0eb4d5ce048, 3a5d646362b84df9be83baaa97b0f601; no voters:
I20260807 00:39:57.688691 3950 raft_consensus.cc:2781] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:57.688836 3950 raft_consensus.cc:686] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [term 1 LEADER]: Becoming Leader. State: Replica: 3a5d646362b84df9be83baaa97b0f601, State: Running, Role: LEADER
I20260807 00:39:57.688907 3950 consensus_queue.cc:237] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } }
I20260807 00:39:57.689337 3998 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: TxnStatusTablet state changed. Reason: New leader 3a5d646362b84df9be83baaa97b0f601. Latest consensus state: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.689545 3998 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:39:57.689833 4000 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 00:39:57.689854 3208 catalog_manager.cc:5495] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131). New cstate: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } health_report { overall_health: HEALTHY } } }
I20260807 00:39:57.689949 4000 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 00:39:57.705374 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.702414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33324 (local address 127.1.116.131:38057)
0807 00:39:57.702500 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.702502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.702514 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:39:57.702633 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.702634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.702660 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.702731 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:39:57.702916 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.703488 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.704390 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.704616 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.704713 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.704931 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:39:57.705086 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:39:57.705196 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.705260 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.705288 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:39:57.705426 3990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.702324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:38057 (local address 127.0.0.1:33324)
0807 00:39:57.702413 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.702427 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.702491 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.702745 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.702746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.702748 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:39:57.702818 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.702824 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.703592 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.703593 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.704277 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.704279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.704715 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.704716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.704784 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.704809 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:39:57.705176 (+ 367us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:39:57.705188 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.705255 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.705328 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 00:39:57.706840 3431 raft_consensus.cc:1261] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Refusing update from remote peer 3a5d646362b84df9be83baaa97b0f601: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:39:57.706990 3624 raft_consensus.cc:1261] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Refusing update from remote peer 3a5d646362b84df9be83baaa97b0f601: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:39:57.707130 3950 consensus_queue.cc:1035] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:39:57.707283 3998 consensus_queue.cc:1035] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601 [LEADER]: Connected to new peer: Peer: permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:39:57.707978 3956 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: TxnStatusTablet state changed. Reason: New leader 3a5d646362b84df9be83baaa97b0f601. Latest consensus state: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.708009 3949 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: TxnStatusTablet state changed. Reason: New leader 3a5d646362b84df9be83baaa97b0f601. Latest consensus state: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.708122 3949 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:39:57.708109 3956 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:39:57.709237 3998 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.709329 3998 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:39:57.709378 3998 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.709419 3998 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 3a5d646362b84df9be83baaa97b0f601: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:39:57.709468 3961 mvcc.cc:205] Tried to move back new op lower bound from 7315714857804496896 to 7315714857734508544. Current Snapshot: MvccSnapshot[applied={T|T < 7315714857804496896}]
I20260807 00:39:57.709602 3956 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.709681 3956 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 22bf67686fca458b9ccbecd0bc33861d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:39:57.709637 3949 tablet_replica.cc:439] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a5d646362b84df9be83baaa97b0f601" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 } } peers { permanent_uuid: "22bf67686fca458b9ccbecd0bc33861d" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 39923 } } peers { permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 } } }
I20260807 00:39:57.709721 3949 tablet_replica.cc:442] T 0a4b243162cc4daab0f2dd98518ebf43 P 1c818a15c8d547df8051e0eb4d5ce048: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:39:57.717801 3939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.714699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:56537 (local address 127.1.116.131:38057)
0807 00:39:57.714776 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.714778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.714934 (+ 156us) server_negotiation.cc:407] Connection header received
0807 00:39:57.714987 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.714988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.715000 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.715071 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:39:57.715197 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.715943 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.716932 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.717234 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.717317 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.717482 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:39:57.717519 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:39:57.717580 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.717677 (+ 97us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.717699 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:39:57.717829 4010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.714631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:38057 (local address 127.1.116.130:56537)
0807 00:39:57.714823 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.714835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.714931 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.715071 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.715072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.715073 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:39:57.715126 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.715128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.716034 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.716035 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.716838 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.716841 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.717326 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.717327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.717375 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.717390 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:39:57.717591 (+ 201us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:39:57.717593 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.717665 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.717715 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":113,"threads_started":1}
I20260807 00:39:57.720862 3838 raft_consensus.cc:1261] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Refusing update from remote peer 22bf67686fca458b9ccbecd0bc33861d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:39:57.721035 3431 raft_consensus.cc:1261] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Refusing update from remote peer 22bf67686fca458b9ccbecd0bc33861d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:39:57.721306 3956 consensus_queue.cc:1035] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c818a15c8d547df8051e0eb4d5ce048" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35509 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:39:57.721462 3952 consensus_queue.cc:1035] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a5d646362b84df9be83baaa97b0f601" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 38057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:39:57.733932 3937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.729242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:42705 (local address 127.1.116.130:39923)
0807 00:39:57.729291 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.729293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.730892 (+ 1599us) server_negotiation.cc:407] Connection header received
0807 00:39:57.730959 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.730960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.730972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.731024 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:39:57.731337 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.732087 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.732930 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.733237 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.733317 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.733654 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:39:57.733693 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:39:57.733767 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.733838 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.733860 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:39:57.733973 4015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.728952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:39923 (local address 127.1.116.131:42705)
0807 00:39:57.730801 (+ 1849us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:57.730814 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:57.730906 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:57.731025 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:57.731026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:57.731084 (+ 58us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:39:57.731224 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.731226 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.732199 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.732200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.732836 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:57.732838 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.733462 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.733463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:57.733530 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.733548 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:39:57.733770 (+ 222us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:39:57.733771 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 00:39:57.733802 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 00:39:57.733845 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1797,"thread_start_us":106,"threads_started":1}
I20260807 00:39:57.743544 3208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:39:57.743649 3208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34470:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:39:57.744539 3208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9220eb3d5ed4d468e4b8b933f25adf8: Sending DeleteTablet for 3 replicas of tablet 3659a986bc8342e6ac88415a946ca3fc
I20260807 00:39:57.744894 3811 tablet_service.cc:1514] Processing DeleteTablet for tablet 3659a986bc8342e6ac88415a946ca3fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:39:57 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:33310
I20260807 00:39:57.744969 3402 tablet_service.cc:1514] Processing DeleteTablet for tablet 3659a986bc8342e6ac88415a946ca3fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:39:57 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:41322
I20260807 00:39:57.745038 3974 tablet_replica.cc:330] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: stopping tablet replica
I20260807 00:39:57.745079 3972 tablet_replica.cc:330] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: stopping tablet replica
I20260807 00:39:57.745133 3972 raft_consensus.cc:2229] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:39:57.745129 3974 raft_consensus.cc:2229] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:39:57.745173 3974 raft_consensus.cc:2258] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:39:57.745172 3972 raft_consensus.cc:2258] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:39:57.745507 3972 ts_tablet_manager.cc:1905] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:39:57.745520 3974 ts_tablet_manager.cc:1905] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:39:57.747059 3974 ts_tablet_manager.cc:1918] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:39:57.747114 3974 log.cc:1196] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/3659a986bc8342e6ac88415a946ca3fc
I20260807 00:39:57.747146 3972 ts_tablet_manager.cc:1918] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:39:57.747205 3972 log.cc:1196] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/3659a986bc8342e6ac88415a946ca3fc
I20260807 00:39:57.747298 3974 ts_tablet_manager.cc:1939] T 3659a986bc8342e6ac88415a946ca3fc P 3a5d646362b84df9be83baaa97b0f601: Deleting consensus metadata
I20260807 00:39:57.747462 3972 ts_tablet_manager.cc:1939] T 3659a986bc8342e6ac88415a946ca3fc P 1c818a15c8d547df8051e0eb4d5ce048: Deleting consensus metadata
I20260807 00:39:57.747727 3592 tablet_service.cc:1514] Processing DeleteTablet for tablet 3659a986bc8342e6ac88415a946ca3fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:39:57 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:57474
I20260807 00:39:57.747804 3184 catalog_manager.cc:4841] TS 1c818a15c8d547df8051e0eb4d5ce048 (127.1.116.129:35509): tablet 3659a986bc8342e6ac88415a946ca3fc (table test-table [id=3706ad20bb6a407c9cdcd73dbcda3b6d]) successfully deleted
I20260807 00:39:57.747918 3973 tablet_replica.cc:330] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: stopping tablet replica
I20260807 00:39:57.747994 3973 raft_consensus.cc:2229] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:39:57.748018 3186 catalog_manager.cc:4841] TS 3a5d646362b84df9be83baaa97b0f601 (127.1.116.131:38057): tablet 3659a986bc8342e6ac88415a946ca3fc (table test-table [id=3706ad20bb6a407c9cdcd73dbcda3b6d]) successfully deleted
I20260807 00:39:57.748068 3973 raft_consensus.cc:2258] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:39:57.748508 3973 ts_tablet_manager.cc:1905] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:39:57.750320 3973 ts_tablet_manager.cc:1918] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:39:57.750394 3973 log.cc:1196] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/3659a986bc8342e6ac88415a946ca3fc
I20260807 00:39:57.750674 3973 ts_tablet_manager.cc:1939] T 3659a986bc8342e6ac88415a946ca3fc P 22bf67686fca458b9ccbecd0bc33861d: Deleting consensus metadata
I20260807 00:39:57.751076 3190 catalog_manager.cc:4841] TS 22bf67686fca458b9ccbecd0bc33861d (127.1.116.130:39923): tablet 3659a986bc8342e6ac88415a946ca3fc (table test-table [id=3706ad20bb6a407c9cdcd73dbcda3b6d]) successfully deleted
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[3139](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063197, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:39:57.761704 3941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.748925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41332 (local address 127.1.116.129:35509)
0807 00:39:57.748971 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.748981 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.749233 (+ 252us) server_negotiation.cc:407] Connection header received
0807 00:39:57.749277 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.749277 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.749291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.749359 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:57.750140 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.750993 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.754476 (+ 3483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.754754 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.754840 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.758510 (+ 3670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:57.758519 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:57.758520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:57.758565 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:57.760358 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.760824 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.760826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.760827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.760882 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.761157 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.761157 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.761158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.761300 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:57.761408 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.761566 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.761639 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
W20260807 00:39:57.762403 3464 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41332
I20260807 00:39:57.770707 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:57.763514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34494 (local address 127.1.116.190:40589)
0807 00:39:57.763568 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:57.763570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:57.763584 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:39:57.763656 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:57.763657 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:57.763669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:57.763750 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:57.764290 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.765088 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.765842 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:57.766148 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:57.766234 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:57.767521 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:57.767523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:57.767523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:57.767532 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:57.769355 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.769736 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.769737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.769738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.769809 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:57.770040 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:57.770041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:57.770041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:57.770186 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:57.770312 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:57.770498 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 00:39:57.770609 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260807 00:39:57.770956 3208 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34494
I20260807 00:39:57.771533 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 3310
I20260807 00:39:57.790498 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 3504
I20260807 00:39:57.804364 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 3705
I20260807 00:39:57.817299 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 3158
2026-08-07T00:39:57Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2806 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[4030](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[4030](info): set up 2 sockets
Aug 07 00:39:57 dist-test-slave-h6n0 krb5kdc[4030](info): commencing operation
krb5kdc: starting...
W20260807 00:39:59.088112 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:39:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:39:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:39:59.220916 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43447
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:43447
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:59.306388 4049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:59.306613 4049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:59.306638 4049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:59.307859 4049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:39:59.307906 4049 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 00:39:59.307920 4049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:59.307934 4049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:59.307969 4049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:39:59.308004 4049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:39:59.309500 4049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:43447
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:59.309736 4049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:59.309988 4049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:59.315069 4055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:59.315060 4058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:59.315323 4049 server_base.cc:1034] running on GCE node
W20260807 00:39:59.315081 4056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:59.315768 4049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:59.316176 4049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:59.317337 4049 hybrid_clock.cc:648] HybridClock initialized: now 1786063199317326 us; error 32 us; skew 500 ppm
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:59.323175 4049 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:39:59.323650 4049 webserver.cc:460] Webserver started at http://127.1.116.190:36041/ using document root <none> and password file <none>
I20260807 00:39:59.323887 4049 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:59.323933 4049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:59.324034 4049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:59.325206 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "07a1a69943797c771ef1690bc9397ad9"
server_key_iv: "cfb4ef00c81838a2bb2499088b30d8ee"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.325553 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "07a1a69943797c771ef1690bc9397ad9"
server_key_iv: "cfb4ef00c81838a2bb2499088b30d8ee"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.327093 4049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:39:59.327899 4066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:59.328184 4049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:59.328311 4049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "07a1a69943797c771ef1690bc9397ad9"
server_key_iv: "cfb4ef00c81838a2bb2499088b30d8ee"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.328400 4049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:59.338371 4049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:59.340718 4049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:59.340880 4049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:59.346951 4049 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:43447
I20260807 00:39:59.347056 4159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:43447 every 8 connection(s)
I20260807 00:39:59.347398 4049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:39:59.348135 4160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:59.350385 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Bootstrap starting.
I20260807 00:39:59.351046 4160 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:59.351294 4160 log.cc:824] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:59.352052 4160 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: No bootstrap required, opened a new log
I20260807 00:39:59.354220 4160 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43447 } }
I20260807 00:39:59.354363 4160 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:59.354385 4160 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b797c3b8cdb4dfabeb9e08a4f4f9d8e, State: Initialized, Role: FOLLOWER
I20260807 00:39:59.354527 4160 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43447 } }
I20260807 00:39:59.354636 4160 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:39:59.354660 4160 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:39:59.354688 4160 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:39:59.355166 4160 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43447 } }
I20260807 00:39:59.355257 4160 leader_election.cc:302] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b797c3b8cdb4dfabeb9e08a4f4f9d8e; no voters:
I20260807 00:39:59.355402 4160 leader_election.cc:288] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:39:59.355434 4163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:39:59.355567 4163 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [term 1 LEADER]: Becoming Leader. State: Replica: 3b797c3b8cdb4dfabeb9e08a4f4f9d8e, State: Running, Role: LEADER
I20260807 00:39:59.355654 4163 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43447 } }
I20260807 00:39:59.355690 4160 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:39:59.356091 4168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b797c3b8cdb4dfabeb9e08a4f4f9d8e. Latest consensus state: current_term: 1 leader_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43447 } } }
I20260807 00:39:59.356124 4165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b797c3b8cdb4dfabeb9e08a4f4f9d8e" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43447 } } }
I20260807 00:39:59.356231 4168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:59.356237 4165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e [sys.catalog]: This master's current role is: LEADER
I20260807 00:39:59.356801 4180 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:39:59.357167 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 4049
I20260807 00:39:59.357263 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:39:59.357450 1490 external_mini_cluster.cc:1428] Setting key 2d8b8cb36953565d34db4321e31350f3
I20260807 00:39:59.357570 4180 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:39:59.360121 4180 catalog_manager.cc:1347] Generated new cluster ID: 64d7b1d584714a039837657122f78231
I20260807 00:39:59.360175 4180 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:59.370294 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.358110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53324 (local address 127.1.116.190:43447)
0807 00:39:59.358418 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.358427 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.358459 (+ 32us) server_negotiation.cc:407] Connection header received
0807 00:39:59.358619 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.358626 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.358687 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.358830 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.359633 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.361094 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.362269 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.362905 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.363038 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.366182 (+ 3144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.366188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.366189 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.366204 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.368372 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.368829 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.368832 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.368832 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.368944 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.369181 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.369182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.369182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.369539 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.369704 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.369863 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.370025 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":96,"threads_started":1}
I20260807 00:39:59.370792 4180 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:39:59.371451 4180 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:39:59.379413 4180 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Generated new TSK 0
I20260807 00:39:59.379594 4180 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:39:59.409297 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43447
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:59.500137 4202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:59.500496 4202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:59.500540 4202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:59.501719 4202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:59.501780 4202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:59.501814 4202 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 00:39:59.501847 4202 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:39:59.501884 4202 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:39:59.503391 4202 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:59.503690 4202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:59.503948 4202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:59.509414 4214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:59.509413 4211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:59.509474 4208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:59.509858 4202 server_base.cc:1034] running on GCE node
I20260807 00:39:59.510106 4202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:59.510386 4202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:59.511538 4202 hybrid_clock.cc:648] HybridClock initialized: now 1786063199511531 us; error 48 us; skew 500 ppm
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:59.514334 4202 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:39:59.514895 4202 webserver.cc:460] Webserver started at http://127.1.116.129:45069/ using document root <none> and password file <none>
I20260807 00:39:59.515162 4202 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:59.515232 4202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:59.515390 4202 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:59.516559 4202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "eb00dea7540e47178d64e5236cdc1302"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "55dcfdcfd8912c34716c78365edf05f2"
server_key_iv: "e429c1d1acf6a989a4155fd78a5a5dd9"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.516976 4202 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "eb00dea7540e47178d64e5236cdc1302"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "55dcfdcfd8912c34716c78365edf05f2"
server_key_iv: "e429c1d1acf6a989a4155fd78a5a5dd9"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.518456 4202 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:39:59.519325 4223 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:59.519536 4202 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:59.519647 4202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "eb00dea7540e47178d64e5236cdc1302"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "55dcfdcfd8912c34716c78365edf05f2"
server_key_iv: "e429c1d1acf6a989a4155fd78a5a5dd9"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.519728 4202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:59.527606 4202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:59.529560 4202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:59.529721 4202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:59.530388 4202 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:59.530445 4202 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:59.530501 4202 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:59.530516 4202 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:59.539776 4202 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:37487
I20260807 00:39:59.540144 4202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:39:59.540136 4391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:37487 every 8 connection(s)
I20260807 00:39:59.542598 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.530868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:53237 (local address 127.1.116.190:43447)
0807 00:39:59.530945 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.530949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.531236 (+ 287us) server_negotiation.cc:407] Connection header received
0807 00:39:59.532100 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.532102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.532117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.532165 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.533207 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.533725 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.535558 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.535825 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.535902 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.539436 (+ 3534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.539438 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.539438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.539447 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.541192 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.541611 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.541613 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.541613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.541669 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.541917 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.541918 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.541919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.542042 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.542119 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.542395 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.542478 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 00:39:59.542821 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.530848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43447 (local address 127.1.116.129:53237)
0807 00:39:59.531060 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.531106 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.531971 (+ 865us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.532260 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.532266 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.532735 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:59.533074 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.533080 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.533824 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.533825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.535463 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.535467 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.535919 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.535920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.536096 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.536727 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:59.536739 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:59.539317 (+ 2578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:59.541290 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.541297 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.541299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.541535 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.541741 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.541742 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.541743 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.541851 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.542110 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:59.542118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:59.542264 (+ 146us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.542372 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.542582 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":120,"threads_started":1}
I20260807 00:39:59.543471 4394 heartbeater.cc:347] Connected to a master server at 127.1.116.190:43447
I20260807 00:39:59.543565 4394 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:59.543779 4394 heartbeater.cc:511] Master 127.1.116.190:43447 requested a full tablet report, sending...
I20260807 00:39:59.544438 4096 ts_manager.cc:194] Registered new tserver with Master: eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487)
I20260807 00:39:59.545173 4096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:53237
I20260807 00:39:59.545581 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 4202
I20260807 00:39:59.545699 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:39:59.545897 1490 external_mini_cluster.cc:1428] Setting key 7ff6d7e5f2bb061e5b46521c74f52fd8
I20260807 00:39:59.551192 4404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.544620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43447 (local address 127.1.116.129:59197)
0807 00:39:59.544795 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.544806 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.544883 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.545022 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.545023 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.545203 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:59.545283 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.545286 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.546320 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.546321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.546898 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.546900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.547288 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.547289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.547368 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.547787 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:59.547791 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:59.548061 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:59.549973 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.549974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.549975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.550187 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.550435 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.550436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.550436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.550512 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.550886 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:59.550887 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:59.550935 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.551003 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.551113 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260807 00:39:59.551275 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.544617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:59197 (local address 127.1.116.190:43447)
0807 00:39:59.544658 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.544659 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.544877 (+ 218us) server_negotiation.cc:407] Connection header received
0807 00:39:59.544942 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.544943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.544953 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.544996 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.545384 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.546122 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.546987 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.547202 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.547286 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.548174 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.548176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.548176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.548182 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.549860 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.550267 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.550268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.550268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.550327 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.550651 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.550652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.550652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.550792 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.550873 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.551051 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.551141 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:39:59.573170 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43447
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:59.659297 4408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:59.659482 4408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:59.659507 4408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:59.660547 4408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:59.660585 4408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:59.660605 4408 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 00:39:59.660617 4408 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:39:59.660634 4408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:39:59.661954 4408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:59.662125 4408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:59.662302 4408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:59.667726 4416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:59.667732 4414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:59.667790 4413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:59.667856 4408 server_base.cc:1034] running on GCE node
I20260807 00:39:59.668233 4408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:59.668473 4408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:59.669643 4408 hybrid_clock.cc:648] HybridClock initialized: now 1786063199669635 us; error 42 us; skew 500 ppm
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:59.672549 4408 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:39:59.673156 4408 webserver.cc:460] Webserver started at http://127.1.116.130:36641/ using document root <none> and password file <none>
I20260807 00:39:59.673352 4408 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:59.673398 4408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:59.673499 4408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:59.674443 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "e8b670e43eb146d3a676debb1e881a86"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "f823fd8712847a04e4f266c8a02a1d52"
server_key_iv: "08fac0d8f8070befccde4dda83c3db81"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.674826 4408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "e8b670e43eb146d3a676debb1e881a86"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "f823fd8712847a04e4f266c8a02a1d52"
server_key_iv: "08fac0d8f8070befccde4dda83c3db81"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.676189 4408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:59.677059 4423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:59.677285 4408 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:39:59.677397 4408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "e8b670e43eb146d3a676debb1e881a86"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "f823fd8712847a04e4f266c8a02a1d52"
server_key_iv: "08fac0d8f8070befccde4dda83c3db81"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.677479 4408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:59.688901 4408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:59.691244 4408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:59.691367 4408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:59.691888 4408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:59.691932 4408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:59.691954 4408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:59.691967 4408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:59.700503 4408 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:44821
I20260807 00:39:59.700736 4608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:44821 every 8 connection(s)
I20260807 00:39:59.700978 4408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:39:59.703845 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.692414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:50453 (local address 127.1.116.190:43447)
0807 00:39:59.692509 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.692512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.692673 (+ 161us) server_negotiation.cc:407] Connection header received
0807 00:39:59.693122 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.693123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.693142 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.693251 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.694074 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.694985 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.696652 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.696993 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.697084 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.700790 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.700791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.700792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.700800 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.702497 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.702973 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.702974 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.702975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.703031 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.703269 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.703270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.703270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.703384 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.703440 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.703702 (+ 262us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.703774 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:39:59.703944 4442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.692331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43447 (local address 127.1.116.130:50453)
0807 00:39:59.692533 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.692571 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.693008 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.693313 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.693316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.693654 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:59.693957 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.693964 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.695110 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.695111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.696533 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.696536 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.697114 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.697116 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.697265 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.697887 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:59.697897 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:59.700581 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:59.702686 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.702691 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.702693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.702900 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.703102 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.703102 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.703103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.703205 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.703446 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:59.703450 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:59.703562 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.703647 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.703760 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":97,"threads_started":1}
I20260807 00:39:59.704466 4609 heartbeater.cc:347] Connected to a master server at 127.1.116.190:43447
I20260807 00:39:59.704559 4609 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:59.704763 4609 heartbeater.cc:511] Master 127.1.116.190:43447 requested a full tablet report, sending...
I20260807 00:39:59.705216 4095 ts_manager.cc:194] Registered new tserver with Master: e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821)
I20260807 00:39:59.705821 4095 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:50453
I20260807 00:39:59.709203 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 4408
I20260807 00:39:59.709309 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:39:59.709520 1490 external_mini_cluster.cc:1428] Setting key d209d7ad38ae502eced84ce28a003778
I20260807 00:39:59.713899 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.705628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:51911 (local address 127.1.116.190:43447)
0807 00:39:59.705680 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.705681 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.705752 (+ 71us) server_negotiation.cc:407] Connection header received
0807 00:39:59.705827 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.705828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.705836 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.705913 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.706332 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.706947 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.707777 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.707985 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.708058 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.709059 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.709061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.709061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.709067 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.712699 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.713038 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.713040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.713040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.713113 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.713321 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.713322 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.713323 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.713474 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.713548 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.713723 (+ 175us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.713809 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:39:59.713912 4617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.705523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43447 (local address 127.1.116.130:51911)
0807 00:39:59.705682 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.705695 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.705760 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.705944 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.705945 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.706143 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:59.706228 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.706230 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.707046 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.707046 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.707661 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.707666 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.708106 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.708107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.708194 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.708643 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:59.708648 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:59.708952 (+ 304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:59.712789 (+ 3837us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.712790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.712791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.712963 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.713185 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.713186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.713186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.713252 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.713549 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:59.713550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:59.713605 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.713696 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.713809 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":93,"threads_started":1}
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:39:59.737629 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43447
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:39:59.829027 4620 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:39:59.829300 4620 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:39:59.829340 4620 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:39:59.830727 4620 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:39:59.830785 4620 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:39:59.830821 4620 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 00:39:59.830847 4620 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:39:59.830896 4620 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:39:59.832458 4620 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4620
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:39:59.832736 4620 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:39:59.832973 4620 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:39:59.838450 4634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:59.838485 4636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:39:59.838466 4630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:39:59.838907 4620 server_base.cc:1034] running on GCE node
I20260807 00:39:59.839152 4620 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:39:59.839434 4620 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:39:59.840600 4620 hybrid_clock.cc:648] HybridClock initialized: now 1786063199840585 us; error 39 us; skew 500 ppm
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:39:59.843355 4620 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:39:59.843773 4620 webserver.cc:460] Webserver started at http://127.1.116.131:40849/ using document root <none> and password file <none>
I20260807 00:39:59.844102 4620 fs_manager.cc:359] Metadata directory not provided
I20260807 00:39:59.844180 4620 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:39:59.844324 4620 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:39:59.845430 4620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "6c794029feaa4e04b5b99f178a325e79"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "5b837f67089693a24eca5dbef2b33351"
server_key_iv: "6c081cc2a9583d05c48118fc23e7e786"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.845822 4620 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "6c794029feaa4e04b5b99f178a325e79"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "5b837f67089693a24eca5dbef2b33351"
server_key_iv: "6c081cc2a9583d05c48118fc23e7e786"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.847379 4620 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:39:59.848254 4647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:59.848495 4620 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:39:59.848610 4620 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "6c794029feaa4e04b5b99f178a325e79"
format_stamp: "Formatted at 2026-08-07 00:39:59 on dist-test-slave-h6n0"
server_key: "5b837f67089693a24eca5dbef2b33351"
server_key_iv: "6c081cc2a9583d05c48118fc23e7e786"
server_key_version: "encryptionkey@0"
I20260807 00:39:59.848685 4620 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:39:59.855626 4620 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:39:59.857753 4620 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:39:59.857899 4620 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:39:59.858407 4620 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:39:59.858472 4620 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:39:59.858520 4620 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:39:59.858568 4620 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:59.866888 4620 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:32803
I20260807 00:39:59.866986 4826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:32803 every 8 connection(s)
I20260807 00:39:59.867334 4620 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:39:59.869586 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.859036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:59539 (local address 127.1.116.190:43447)
0807 00:39:59.859122 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.859126 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.859417 (+ 291us) server_negotiation.cc:407] Connection header received
0807 00:39:59.860395 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.860396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.860409 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.860502 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.861393 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.862272 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.863468 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.863722 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.863792 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.866291 (+ 2499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.866293 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.866293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.866301 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.868029 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.868553 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.868554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.868554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.868615 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.868894 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.868895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.868896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.869043 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.869189 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.869428 (+ 239us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.869519 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:39:59.869776 4671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.859021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43447 (local address 127.1.116.131:59539)
0807 00:39:59.859263 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.859315 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.860254 (+ 939us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.860598 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.860604 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.860992 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:59.861281 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.861287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.862368 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.862369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.863376 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.863379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.863889 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.863890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.864035 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.864615 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:59.864627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:59.866189 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:59.868153 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.868158 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.868160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.868474 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.868685 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.868686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.868686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.868791 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.869179 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:59.869184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:59.869304 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.869395 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.869530 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":141,"threads_started":1}
I20260807 00:39:59.870316 4827 heartbeater.cc:347] Connected to a master server at 127.1.116.190:43447
I20260807 00:39:59.870431 4827 heartbeater.cc:464] Registering TS with master...
I20260807 00:39:59.870685 4827 heartbeater.cc:511] Master 127.1.116.190:43447 requested a full tablet report, sending...
I20260807 00:39:59.871131 4096 ts_manager.cc:194] Registered new tserver with Master: 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803)
I20260807 00:39:59.871655 4096 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:59539
I20260807 00:39:59.873515 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 4620
I20260807 00:39:59.873607 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:39:59.873793 1490 external_mini_cluster.cc:1428] Setting key 71a9554d22bcb98864e07794d899197b
I20260807 00:39:59.874854 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:39:59.877804 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.871488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:42567 (local address 127.1.116.190:43447)
0807 00:39:59.871528 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.871530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.871749 (+ 219us) server_negotiation.cc:407] Connection header received
0807 00:39:59.871775 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.871776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.871788 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.871853 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.872342 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.872839 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.873763 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.874041 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.874111 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.875186 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.875188 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.875188 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.875195 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.876664 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.877029 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.877030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.877031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.877085 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.877284 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.877285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.877285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.877394 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.877468 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.877623 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.877708 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 00:39:59.877811 4840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.871410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43447 (local address 127.1.116.131:42567)
0807 00:39:59.871543 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.871628 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.871702 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.871990 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.871991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.872176 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:39:59.872254 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.872256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.872934 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.872935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.873627 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.873630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.874126 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.874127 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.874209 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.874700 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:39:59.874706 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:39:59.875090 (+ 384us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:39:59.876768 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.876769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.876770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.876944 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.877158 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:39:59.877159 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:39:59.877159 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:39:59.877217 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:39:59.877460 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:39:59.877461 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:39:59.877509 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.877584 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.877711 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":81,"threads_started":1}
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:39:59 dist-test-slave-h6n0 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:39:59.901221 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.893814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53332 (local address 127.1.116.190:43447)
0807 00:39:59.893880 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.893882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.893891 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:39:59.893913 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.893914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.893923 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.893989 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:39:59.894735 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.895233 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.896026 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.896205 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.896265 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.898767 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:39:59.898768 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:39:59.898768 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:39:59.898774 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:39:59.900116 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.900479 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.900479 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.900480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.900529 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:39:59.900720 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:39:59.900721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:39:59.900721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:39:59.900817 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:39:59.900920 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.901079 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.901142 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 00:39:59.903188 4096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:39:59.905242 4096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:39:59.913033 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.908211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:44821 (local address 127.0.0.1:36418)
0807 00:39:59.908541 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.908556 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.908638 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.909060 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.909063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.909065 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:59.909147 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.909149 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.910139 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.910140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.911687 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.911691 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.912609 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.912610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.912716 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.912722 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.912840 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.912898 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":3,"thread_start_us":53,"threads_started":1}
I20260807 00:39:59.913218 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.908322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.1.116.131:32803)
0807 00:39:59.908788 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.908791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.908802 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:39:59.908845 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.908847 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.908865 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.908946 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:59.909273 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.910410 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.911915 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.912626 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.912693 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.912742 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.913074 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.913116 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":114,"threads_started":1}
I20260807 00:39:59.913300 4864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.908426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37487 (local address 127.0.0.1:48750)
0807 00:39:59.908847 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.908859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.908914 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.909176 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.909178 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.909181 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:59.909245 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.909248 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.910517 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.910518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.911564 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.911567 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.912605 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.912606 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.912697 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.912712 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.912792 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.912856 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":1,"thread_start_us":82,"threads_started":1}
I20260807 00:39:59.913466 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.908475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36418 (local address 127.1.116.130:44821)
0807 00:39:59.908671 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.908677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.908691 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:39:59.908869 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.908873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.908894 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.908978 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:59.909249 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.910040 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.911955 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.912512 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.912606 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.912681 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.913293 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.913349 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":86,"threads_started":1}
I20260807 00:39:59.913619 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.908209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:32803 (local address 127.0.0.1:51162)
0807 00:39:59.908473 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 00:39:59.908520 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:39:59.908638 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:39:59.909058 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:39:59.909062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:39:59.909065 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:39:59.909196 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.909198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.910524 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.910546 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.911822 (+ 1276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:39:59.911825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.912782 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.912783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:39:59.912848 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.912853 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:39:59.912901 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 00:39:59.912963 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":39,"spinlock_wait_cycles":1792,"thread_start_us":105,"threads_started":1}
I20260807 00:39:59.913089 4867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:39:59.908666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48750 (local address 127.1.116.129:37487)
0807 00:39:59.909022 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0807 00:39:59.909026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:39:59.909043 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:39:59.909080 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:39:59.909082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:39:59.909092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:39:59.909176 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:39:59.909334 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.910421 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.911655 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:39:59.912432 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:39:59.912507 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:39:59.912564 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:39:59.912876 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 00:39:59.912948 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":90,"threads_started":1}
I20260807 00:39:59.914228 4511 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac5598f615e4107a71cd7496805d0a2 (DEFAULT_TABLE table=test-table [id=b92509687f6d4a4aab6f69cd2af1c79f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:59.914433 4727 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac5598f615e4107a71cd7496805d0a2 (DEFAULT_TABLE table=test-table [id=b92509687f6d4a4aab6f69cd2af1c79f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:59.914719 4511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac5598f615e4107a71cd7496805d0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:59.914772 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac5598f615e4107a71cd7496805d0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:59.914584 4298 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac5598f615e4107a71cd7496805d0a2 (DEFAULT_TABLE table=test-table [id=b92509687f6d4a4aab6f69cd2af1c79f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:39:59.915045 4298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac5598f615e4107a71cd7496805d0a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:39:59.918190 4868 tablet_bootstrap.cc:492] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Bootstrap starting.
I20260807 00:39:59.918640 4871 tablet_bootstrap.cc:492] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Bootstrap starting.
I20260807 00:39:59.919514 4871 tablet_bootstrap.cc:654] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:59.919652 4868 tablet_bootstrap.cc:654] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:59.919991 4871 log.cc:824] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:59.920050 4868 log.cc:824] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:59.920405 4870 tablet_bootstrap.cc:492] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Bootstrap starting.
I20260807 00:39:59.920992 4871 tablet_bootstrap.cc:492] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: No bootstrap required, opened a new log
I20260807 00:39:59.921094 4870 tablet_bootstrap.cc:654] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Neither blocks nor log segments found. Creating new log.
I20260807 00:39:59.921087 4871 ts_tablet_manager.cc:1397] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:39:59.921150 4868 tablet_bootstrap.cc:492] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: No bootstrap required, opened a new log
I20260807 00:39:59.921271 4868 ts_tablet_manager.cc:1397] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 00:39:59.921465 4870 log.cc:824] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Log is configured to *not* fsync() on all Append() calls
I20260807 00:39:59.922713 4870 tablet_bootstrap.cc:492] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: No bootstrap required, opened a new log
I20260807 00:39:59.922791 4870 ts_tablet_manager.cc:1397] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:39:59.923468 4868 raft_consensus.cc:348] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:39:59.923687 4868 raft_consensus.cc:374] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:59.923740 4868 raft_consensus.cc:729] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c794029feaa4e04b5b99f178a325e79, State: Initialized, Role: FOLLOWER
I20260807 00:39:59.923873 4868 consensus_queue.cc:260] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:39:59.923796 4871 raft_consensus.cc:348] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:39:59.924038 4871 raft_consensus.cc:374] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:59.924096 4871 raft_consensus.cc:729] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b670e43eb146d3a676debb1e881a86, State: Initialized, Role: FOLLOWER
I20260807 00:39:59.924211 4868 ts_tablet_manager.cc:1428] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260807 00:39:59.924253 4871 consensus_queue.cc:260] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:39:59.924355 4827 heartbeater.cc:503] Master 127.1.116.190:43447 was elected leader, sending a full tablet report...
I20260807 00:39:59.924687 4609 heartbeater.cc:503] Master 127.1.116.190:43447 was elected leader, sending a full tablet report...
I20260807 00:39:59.925221 4871 ts_tablet_manager.cc:1428] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 00:39:59.925166 4870 raft_consensus.cc:348] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:39:59.925335 4870 raft_consensus.cc:374] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:39:59.925413 4870 raft_consensus.cc:729] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Initialized, Role: FOLLOWER
I20260807 00:39:59.925537 4870 consensus_queue.cc:260] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:39:59.925868 4870 ts_tablet_manager.cc:1428] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:39:59.925951 4394 heartbeater.cc:503] Master 127.1.116.190:43447 was elected leader, sending a full tablet report...
W20260807 00:39:59.951680 4611 tablet.cc:2369] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:40:00.041154 4396 tablet.cc:2369] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:40:00.118036 4830 tablet.cc:2369] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:00.261547 4879 raft_consensus.cc:482] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:00.261755 4879 raft_consensus.cc:504] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:40:00.262326 4879 leader_election.cc:288] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821), 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803)
I20260807 00:40:00.266498 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.262706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:40281 (local address 127.1.116.130:44821)
0807 00:40:00.262851 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.262854 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.262906 (+ 52us) server_negotiation.cc:407] Connection header received
0807 00:40:00.263022 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.263023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.263050 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.263125 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:00.263337 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.264027 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.265605 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.266024 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.266154 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.266166 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.266349 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.266419 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260807 00:40:00.266495 4881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.262644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:44821 (local address 127.1.116.129:40281)
0807 00:40:00.262814 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.262830 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.262940 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.263149 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.263150 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.263153 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:00.263256 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.263259 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.264128 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.264130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.265484 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.265487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.266174 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.266175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.266251 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.266256 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.266336 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.266386 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":107,"thread_start_us":85,"threads_started":1}
I20260807 00:40:00.267095 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.262637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:32803 (local address 127.1.116.129:56765)
0807 00:40:00.262790 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.262802 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.262875 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.263082 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.263083 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.263087 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:00.263181 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.263184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.264168 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.264169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.265484 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.265487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.266815 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.266816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.266894 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.266899 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.266995 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.267037 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":15,"thread_start_us":69,"threads_started":1}
I20260807 00:40:00.267179 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.262760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:56765 (local address 127.1.116.131:32803)
0807 00:40:00.262837 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.262840 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.262861 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:40:00.262976 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.262978 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.262990 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.263054 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:00.263267 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.264041 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.266204 (+ 2163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.266698 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.266784 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.266795 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.267028 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.267084 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:40:00.267112 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac5598f615e4107a71cd7496805d0a2" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86" is_pre_election: true
I20260807 00:40:00.267343 4539 raft_consensus.cc:2454] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 0.
I20260807 00:40:00.267529 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac5598f615e4107a71cd7496805d0a2" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c794029feaa4e04b5b99f178a325e79" is_pre_election: true
I20260807 00:40:00.267620 4226 leader_election.cc:302] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8b670e43eb146d3a676debb1e881a86, eb00dea7540e47178d64e5236cdc1302; no voters:
I20260807 00:40:00.267724 4755 raft_consensus.cc:2454] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 0.
I20260807 00:40:00.267781 4879 raft_consensus.cc:2781] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:00.267848 4879 raft_consensus.cc:482] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:00.267884 4879 raft_consensus.cc:3037] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.268795 4879 raft_consensus.cc:504] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:40:00.268977 4879 leader_election.cc:288] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 election: Requested vote from peers e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821), 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803)
I20260807 00:40:00.269132 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac5598f615e4107a71cd7496805d0a2" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86"
I20260807 00:40:00.269201 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac5598f615e4107a71cd7496805d0a2" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c794029feaa4e04b5b99f178a325e79"
I20260807 00:40:00.269223 4539 raft_consensus.cc:3037] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.269323 4755 raft_consensus.cc:3037] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.270138 4539 raft_consensus.cc:2454] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 1.
I20260807 00:40:00.270205 4755 raft_consensus.cc:2454] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 1.
I20260807 00:40:00.270381 4226 leader_election.cc:302] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8b670e43eb146d3a676debb1e881a86, eb00dea7540e47178d64e5236cdc1302; no voters:
I20260807 00:40:00.270500 4879 raft_consensus.cc:2781] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:00.270673 4879 raft_consensus.cc:686] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 1 LEADER]: Becoming Leader. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Running, Role: LEADER
I20260807 00:40:00.270794 4879 consensus_queue.cc:237] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } }
I20260807 00:40:00.274747 4096 catalog_manager.cc:5495] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb00dea7540e47178d64e5236cdc1302 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } health_report { overall_health: HEALTHY } } }
Aug 07 00:40:00 dist-test-slave-h6n0 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:00.383531 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.375643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.1.116.190:43447)
0807 00:40:00.375744 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.375749 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.375877 (+ 128us) server_negotiation.cc:407] Connection header received
0807 00:40:00.375992 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.375993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.376006 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.376132 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:00.376693 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.377231 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.377933 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.378128 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.378214 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.380833 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:00.380835 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:00.380835 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:00.380843 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:00.382355 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:00.382788 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:00.382788 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:00.382789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:00.382844 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:00.383032 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:00.383033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:00.383033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:00.383139 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:00.383226 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.383362 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.383451 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:40:00.383648 4894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.375517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43447 (local address 127.0.0.1:53336)
0807 00:40:00.375771 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.375788 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.375921 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.376140 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.376141 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.376521 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:00.376606 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.376609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.377315 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.377317 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.377850 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.377852 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.378215 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.378216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.378265 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.378918 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:00.378923 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:00.380692 (+ 1769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:00.382447 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:00.382455 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:00.382457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:00.382713 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:00.382912 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:00.382913 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:00.382913 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:00.382984 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:00.383221 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:00.383228 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:00.383284 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.383343 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.383529 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":69,"thread_start_us":129,"threads_started":1}
I20260807 00:40:00.385728 4096 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:53336:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 00:40:00.386020 4096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:00.388108 4298 tablet_service.cc:1467] Processing CreateTablet for tablet 1dcdcae8525a49178e3eaead4c9a3445 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ae0f35eb59146ad8ff20a47461f7679]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:40:00.388093 4511 tablet_service.cc:1467] Processing CreateTablet for tablet 1dcdcae8525a49178e3eaead4c9a3445 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ae0f35eb59146ad8ff20a47461f7679]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:40:00.388095 4727 tablet_service.cc:1467] Processing CreateTablet for tablet 1dcdcae8525a49178e3eaead4c9a3445 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ae0f35eb59146ad8ff20a47461f7679]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:40:00.388290 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dcdcae8525a49178e3eaead4c9a3445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:00.388295 4298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dcdcae8525a49178e3eaead4c9a3445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:00.388295 4511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1dcdcae8525a49178e3eaead4c9a3445. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:00.389804 4868 tablet_bootstrap.cc:492] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: Bootstrap starting.
I20260807 00:40:00.389950 4871 tablet_bootstrap.cc:492] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: Bootstrap starting.
I20260807 00:40:00.389953 4870 tablet_bootstrap.cc:492] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: Bootstrap starting.
I20260807 00:40:00.390337 4868 tablet_bootstrap.cc:654] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:00.390389 4870 tablet_bootstrap.cc:654] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:00.390403 4871 tablet_bootstrap.cc:654] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:00.391166 4868 tablet_bootstrap.cc:492] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: No bootstrap required, opened a new log
I20260807 00:40:00.391239 4868 ts_tablet_manager.cc:1397] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:00.391399 4871 tablet_bootstrap.cc:492] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: No bootstrap required, opened a new log
I20260807 00:40:00.391462 4871 ts_tablet_manager.cc:1397] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 00:40:00.391443 4868 raft_consensus.cc:348] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.391528 4868 raft_consensus.cc:374] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:00.391552 4868 raft_consensus.cc:729] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c794029feaa4e04b5b99f178a325e79, State: Initialized, Role: FOLLOWER
I20260807 00:40:00.391598 4868 consensus_queue.cc:260] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.391633 4871 raft_consensus.cc:348] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.391693 4871 raft_consensus.cc:374] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:00.391726 4868 ts_tablet_manager.cc:1428] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:00.391736 4871 raft_consensus.cc:729] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b670e43eb146d3a676debb1e881a86, State: Initialized, Role: FOLLOWER
I20260807 00:40:00.391781 4871 consensus_queue.cc:260] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.391760 4876 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.391847 4876 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:00.391906 4871 ts_tablet_manager.cc:1428] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:00.391971 4877 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.392131 4877 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:00.392268 4870 tablet_bootstrap.cc:492] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: No bootstrap required, opened a new log
I20260807 00:40:00.392323 4870 ts_tablet_manager.cc:1397] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 00:40:00.392470 4870 raft_consensus.cc:348] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.392567 4870 raft_consensus.cc:374] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:00.392590 4870 raft_consensus.cc:729] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Initialized, Role: FOLLOWER
I20260807 00:40:00.392634 4870 consensus_queue.cc:260] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.392731 4870 ts_tablet_manager.cc:1428] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:00.392854 4879 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.393006 4879 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:00.519449 4879 raft_consensus.cc:482] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:00.519598 4879 raft_consensus.cc:504] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.519903 4879 leader_election.cc:288] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821), 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803)
I20260807 00:40:00.520242 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dcdcae8525a49178e3eaead4c9a3445" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86" is_pre_election: true
I20260807 00:40:00.520219 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dcdcae8525a49178e3eaead4c9a3445" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c794029feaa4e04b5b99f178a325e79" is_pre_election: true
I20260807 00:40:00.520346 4755 raft_consensus.cc:2454] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 0.
I20260807 00:40:00.520358 4539 raft_consensus.cc:2454] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 0.
I20260807 00:40:00.520596 4226 leader_election.cc:302] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8b670e43eb146d3a676debb1e881a86, eb00dea7540e47178d64e5236cdc1302; no voters:
I20260807 00:40:00.520730 4879 raft_consensus.cc:2781] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:00.520787 4879 raft_consensus.cc:482] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:00.520805 4879 raft_consensus.cc:3037] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.521490 4879 raft_consensus.cc:504] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.521656 4879 leader_election.cc:288] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 election: Requested vote from peers e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821), 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803)
I20260807 00:40:00.521868 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dcdcae8525a49178e3eaead4c9a3445" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86"
I20260807 00:40:00.521973 4539 raft_consensus.cc:3037] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.521997 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1dcdcae8525a49178e3eaead4c9a3445" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c794029feaa4e04b5b99f178a325e79"
I20260807 00:40:00.522130 4755 raft_consensus.cc:3037] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.522950 4539 raft_consensus.cc:2454] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 1.
I20260807 00:40:00.523142 4226 leader_election.cc:302] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8b670e43eb146d3a676debb1e881a86, eb00dea7540e47178d64e5236cdc1302; no voters:
I20260807 00:40:00.523200 4755 raft_consensus.cc:2454] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 1.
I20260807 00:40:00.523342 4879 raft_consensus.cc:2781] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:00.523429 4879 raft_consensus.cc:686] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [term 1 LEADER]: Becoming Leader. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Running, Role: LEADER
I20260807 00:40:00.523489 4879 consensus_queue.cc:237] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.523808 4887 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: TxnStatusTablet state changed. Reason: New leader eb00dea7540e47178d64e5236cdc1302. Latest consensus state: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.523927 4887 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:40:00.524291 4902 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 00:40:00.524473 4902 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 00:40:00.524403 4095 catalog_manager.cc:5495] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb00dea7540e47178d64e5236cdc1302 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } health_report { overall_health: UNKNOWN } } }
I20260807 00:40:00.543226 4894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.539328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37487 (local address 127.0.0.1:48762)
0807 00:40:00.539448 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.539469 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.539575 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.539873 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.539874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.539876 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:40:00.539936 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.539939 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.541038 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.541039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.541983 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.541986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.542620 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.542621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.542669 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.542701 (+ 32us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.543059 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.543069 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.543121 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.543164 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 00:40:00.543272 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.539453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48762 (local address 127.1.116.129:37487)
0807 00:40:00.539653 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.539655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.539667 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:00.539722 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.539723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.539750 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.539852 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:00.540053 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.540932 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.542099 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.542449 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.542557 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.542823 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.542975 (+ 152us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.543080 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.543130 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.543159 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260807 00:40:00.545389 4755 raft_consensus.cc:1261] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Refusing update from remote peer eb00dea7540e47178d64e5236cdc1302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:00.545389 4539 raft_consensus.cc:1261] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Refusing update from remote peer eb00dea7540e47178d64e5236cdc1302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:00.545751 4879 consensus_queue.cc:1035] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:00.545883 4887 consensus_queue.cc:1035] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:00.547221 4877 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: TxnStatusTablet state changed. Reason: New leader eb00dea7540e47178d64e5236cdc1302. Latest consensus state: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.547330 4877 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:00.547931 4876 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: TxnStatusTablet state changed. Reason: New leader eb00dea7540e47178d64e5236cdc1302. Latest consensus state: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.548035 4876 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:00.548489 4887 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.548599 4887 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:40:00.548733 4877 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.548827 4877 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P e8b670e43eb146d3a676debb1e881a86: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:00.549079 4908 mvcc.cc:205] Tried to move back new op lower bound from 7315714869430968320 to 7315714869344591872. Current Snapshot: MvccSnapshot[applied={T|T < 7315714869430968320}]
I20260807 00:40:00.550112 4879 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.550199 4879 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P eb00dea7540e47178d64e5236cdc1302: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:40:00.549947 4909 mvcc.cc:205] Tried to move back new op lower bound from 7315714869430968320 to 7315714869344591872. Current Snapshot: MvccSnapshot[applied={T|T < 7315714869430968320}]
I20260807 00:40:00.551410 4876 tablet_replica.cc:439] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } } }
I20260807 00:40:00.551570 4876 tablet_replica.cc:442] T 1dcdcae8525a49178e3eaead4c9a3445 P 6c794029feaa4e04b5b99f178a325e79: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:00.551918 4911 mvcc.cc:205] Tried to move back new op lower bound from 7315714869430968320 to 7315714869344591872. Current Snapshot: MvccSnapshot[applied={T|T < 7315714869430968320}]
I20260807 00:40:00.555855 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.552481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.1.116.129:37487)
0807 00:40:00.552547 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.552549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.552678 (+ 129us) server_negotiation.cc:407] Connection header received
0807 00:40:00.552750 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.552751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.552765 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.552834 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:00.552941 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.553755 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.554993 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.555309 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.555395 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.555571 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.555627 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.555696 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.555751 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.555779 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:40:00.561800 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.558683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37487 (local address 127.1.116.129:45499)
0807 00:40:00.558867 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.558882 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.558944 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.559088 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.559088 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.559090 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:40:00.559150 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.559152 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.560066 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.560067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.560814 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.560818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.561316 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.561317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.561390 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.561404 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.561612 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.561614 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.561655 (+ 41us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.561700 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":107,"threads_started":1}
I20260807 00:40:00.561802 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.558718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:45499 (local address 127.1.116.129:37487)
0807 00:40:00.558794 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.558796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.558957 (+ 161us) server_negotiation.cc:407] Connection header received
0807 00:40:00.558988 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.558989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.559002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.559083 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:00.559210 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.559930 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.560914 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.561226 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.561307 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.561506 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.561543 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.561604 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.561665 (+ 61us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.561699 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:40:00.564402 4755 raft_consensus.cc:1261] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Refusing update from remote peer eb00dea7540e47178d64e5236cdc1302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:00.564410 4539 raft_consensus.cc:1261] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Refusing update from remote peer eb00dea7540e47178d64e5236cdc1302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:00.564661 4924 consensus_queue.cc:1035] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:00.564803 4923 consensus_queue.cc:1035] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:00.577643 4096 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:00.577764 4096 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:00.578711 4096 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Sending DeleteTablet for 3 replicas of tablet 1ac5598f615e4107a71cd7496805d0a2
I20260807 00:40:00.579075 4511 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac5598f615e4107a71cd7496805d0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:00 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:36418
I20260807 00:40:00.579106 4727 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac5598f615e4107a71cd7496805d0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:00 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:51162
I20260807 00:40:00.579214 4296 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac5598f615e4107a71cd7496805d0a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:00 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:48750
I20260807 00:40:00.579494 4931 tablet_replica.cc:330] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: stopping tablet replica
I20260807 00:40:00.579588 4929 tablet_replica.cc:330] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: stopping tablet replica
I20260807 00:40:00.579687 4929 raft_consensus.cc:2229] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:00.579768 4929 raft_consensus.cc:2258] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:00.579733 4096 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53332:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 00:40:00.579706 4931 raft_consensus.cc:2229] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 1 LEADER]: Raft consensus shutting down.
W20260807 00:40:00.579923 4096 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:00.580173 4931 raft_consensus.cc:2258] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:00.580260 4929 ts_tablet_manager.cc:1905] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:00.580665 4931 ts_tablet_manager.cc:1905] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:00.581226 4511 tablet_service.cc:1467] Processing CreateTablet for tablet e78b31d983e549e5b3b282077d5d0d0c (DEFAULT_TABLE table=test-table [id=708b072a1963481e892e5e5a1370bdfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:00.581310 4727 tablet_service.cc:1467] Processing CreateTablet for tablet e78b31d983e549e5b3b282077d5d0d0c (DEFAULT_TABLE table=test-table [id=708b072a1963481e892e5e5a1370bdfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:00.581362 4511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78b31d983e549e5b3b282077d5d0d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:00.581413 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78b31d983e549e5b3b282077d5d0d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:00.582696 4930 tablet_replica.cc:330] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: stopping tablet replica
I20260807 00:40:00.582878 4871 tablet_bootstrap.cc:492] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: Bootstrap starting.
I20260807 00:40:00.583190 4930 raft_consensus.cc:2229] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:00.583280 4930 raft_consensus.cc:2258] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:00.583293 4871 tablet_bootstrap.cc:654] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:00.583375 4868 tablet_bootstrap.cc:492] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: Bootstrap starting.
I20260807 00:40:00.583870 4868 tablet_bootstrap.cc:654] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:00.584192 4871 tablet_bootstrap.cc:492] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: No bootstrap required, opened a new log
I20260807 00:40:00.584231 4871 ts_tablet_manager.cc:1397] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:00.584383 4871 raft_consensus.cc:348] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.584462 4871 raft_consensus.cc:374] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:00.584484 4871 raft_consensus.cc:729] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b670e43eb146d3a676debb1e881a86, State: Initialized, Role: FOLLOWER
I20260807 00:40:00.584513 4296 tablet_service.cc:1467] Processing CreateTablet for tablet e78b31d983e549e5b3b282077d5d0d0c (DEFAULT_TABLE table=test-table [id=708b072a1963481e892e5e5a1370bdfc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:00.584529 4871 consensus_queue.cc:260] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.584616 4296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e78b31d983e549e5b3b282077d5d0d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:00.584640 4871 ts_tablet_manager.cc:1428] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:40:00.584806 4930 ts_tablet_manager.cc:1905] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:00.586057 4868 tablet_bootstrap.cc:492] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: No bootstrap required, opened a new log
I20260807 00:40:00.586110 4868 ts_tablet_manager.cc:1397] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 00:40:00.586190 4870 tablet_bootstrap.cc:492] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: Bootstrap starting.
I20260807 00:40:00.586323 4868 raft_consensus.cc:348] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.586404 4868 raft_consensus.cc:374] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:00.586474 4868 raft_consensus.cc:729] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c794029feaa4e04b5b99f178a325e79, State: Initialized, Role: FOLLOWER
I20260807 00:40:00.586568 4868 consensus_queue.cc:260] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.586630 4870 tablet_bootstrap.cc:654] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:00.586711 4868 ts_tablet_manager.cc:1428] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:00.586385 4930 ts_tablet_manager.cc:1918] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:00.586830 4930 log.cc:1196] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/1ac5598f615e4107a71cd7496805d0a2
I20260807 00:40:00.587118 4930 ts_tablet_manager.cc:1939] T 1ac5598f615e4107a71cd7496805d0a2 P 6c794029feaa4e04b5b99f178a325e79: Deleting consensus metadata
I20260807 00:40:00.587532 4070 catalog_manager.cc:4841] TS 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803): tablet 1ac5598f615e4107a71cd7496805d0a2 (table test-table [id=b92509687f6d4a4aab6f69cd2af1c79f]) successfully deleted
I20260807 00:40:00.589011 4929 ts_tablet_manager.cc:1918] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:00.589095 4929 log.cc:1196] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/1ac5598f615e4107a71cd7496805d0a2
I20260807 00:40:00.589039 4931 ts_tablet_manager.cc:1918] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:00.589265 4931 log.cc:1196] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/1ac5598f615e4107a71cd7496805d0a2
I20260807 00:40:00.589367 4929 ts_tablet_manager.cc:1939] T 1ac5598f615e4107a71cd7496805d0a2 P e8b670e43eb146d3a676debb1e881a86: Deleting consensus metadata
I20260807 00:40:00.589612 4931 ts_tablet_manager.cc:1939] T 1ac5598f615e4107a71cd7496805d0a2 P eb00dea7540e47178d64e5236cdc1302: Deleting consensus metadata
I20260807 00:40:00.589841 4069 catalog_manager.cc:4841] TS e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821): tablet 1ac5598f615e4107a71cd7496805d0a2 (table test-table [id=b92509687f6d4a4aab6f69cd2af1c79f]) successfully deleted
I20260807 00:40:00.590132 4070 catalog_manager.cc:4841] TS eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487): tablet 1ac5598f615e4107a71cd7496805d0a2 (table test-table [id=b92509687f6d4a4aab6f69cd2af1c79f]) successfully deleted
I20260807 00:40:00.590372 4870 tablet_bootstrap.cc:492] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: No bootstrap required, opened a new log
I20260807 00:40:00.590426 4870 ts_tablet_manager.cc:1397] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 00:40:00.590598 4870 raft_consensus.cc:348] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.590679 4870 raft_consensus.cc:374] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:00.590703 4870 raft_consensus.cc:729] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Initialized, Role: FOLLOWER
I20260807 00:40:00.590757 4870 consensus_queue.cc:260] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.590858 4870 ts_tablet_manager.cc:1428] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:00.705026 4876 raft_consensus.cc:482] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:00.705192 4876 raft_consensus.cc:504] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.705703 4876 leader_election.cc:288] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487), e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821)
I20260807 00:40:00.709326 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.706047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:55311 (local address 127.1.116.130:44821)
0807 00:40:00.706103 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.706105 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.706211 (+ 106us) server_negotiation.cc:407] Connection header received
0807 00:40:00.706333 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.706334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.706345 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.706398 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:00.706615 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.707112 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.708652 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.708998 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.709034 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.709048 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.709226 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.709259 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 00:40:00.709352 4936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.705886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:44821 (local address 127.1.116.131:55311)
0807 00:40:00.706129 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.706143 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.706237 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.706402 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.706402 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.706404 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:00.706492 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.706495 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.707226 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.707226 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.708539 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.708543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.709095 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.709095 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.709149 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.709153 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.709218 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.709258 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":3,"thread_start_us":116,"threads_started":1}
I20260807 00:40:00.709769 4935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.705885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37487 (local address 127.1.116.131:39663)
0807 00:40:00.706130 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.706143 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.706204 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.706461 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.706462 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.706463 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:00.706520 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.706521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.707518 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.707519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.708567 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.708569 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.709408 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.709410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.709516 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.709521 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.709583 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.709639 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":116,"threads_started":1}
I20260807 00:40:00.709784 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.706153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:39663 (local address 127.1.116.129:37487)
0807 00:40:00.706199 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.706201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.706214 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:00.706293 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.706294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.706305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.706412 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:00.706623 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.707420 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.708662 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.709270 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.709350 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.709360 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.709620 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.709707 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:40:00.709841 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78b31d983e549e5b3b282077d5d0d0c" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86" is_pre_election: true
I20260807 00:40:00.709931 4539 raft_consensus.cc:2454] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 0.
I20260807 00:40:00.710140 4323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78b31d983e549e5b3b282077d5d0d0c" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb00dea7540e47178d64e5236cdc1302" is_pre_election: true
I20260807 00:40:00.710260 4657 leader_election.cc:302] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c794029feaa4e04b5b99f178a325e79, e8b670e43eb146d3a676debb1e881a86; no voters:
I20260807 00:40:00.710330 4323 raft_consensus.cc:2454] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 0.
I20260807 00:40:00.710404 4876 raft_consensus.cc:2781] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:00.710458 4876 raft_consensus.cc:482] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:00.710481 4876 raft_consensus.cc:3037] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.711565 4876 raft_consensus.cc:504] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.711730 4876 leader_election.cc:288] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 election: Requested vote from peers eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487), e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821)
I20260807 00:40:00.711872 4323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78b31d983e549e5b3b282077d5d0d0c" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb00dea7540e47178d64e5236cdc1302"
I20260807 00:40:00.711966 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e78b31d983e549e5b3b282077d5d0d0c" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86"
I20260807 00:40:00.711994 4323 raft_consensus.cc:3037] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.712021 4539 raft_consensus.cc:3037] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:00.712814 4539 raft_consensus.cc:2454] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 1.
I20260807 00:40:00.712877 4323 raft_consensus.cc:2454] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 1.
I20260807 00:40:00.713013 4657 leader_election.cc:302] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c794029feaa4e04b5b99f178a325e79, e8b670e43eb146d3a676debb1e881a86; no voters:
I20260807 00:40:00.713125 4876 raft_consensus.cc:2781] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:00.713270 4876 raft_consensus.cc:686] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 1 LEADER]: Becoming Leader. State: Replica: 6c794029feaa4e04b5b99f178a325e79, State: Running, Role: LEADER
I20260807 00:40:00.713356 4876 consensus_queue.cc:237] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:00.714187 4095 catalog_manager.cc:5495] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131). New cstate: current_term: 1 leader_uuid: "6c794029feaa4e04b5b99f178a325e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } health_report { overall_health: HEALTHY } } }
I20260807 00:40:00.733752 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.730939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51174 (local address 127.1.116.131:32803)
0807 00:40:00.731009 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.731012 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.731022 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:00.731069 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.731069 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.731100 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.731195 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:00.731350 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.732115 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.732874 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.733148 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.733228 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.733364 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.733525 (+ 161us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.733610 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.733631 (+ 21us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.733659 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:40:00.739571 4907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.737113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:44311 (local address 127.1.116.129:37487)
0807 00:40:00.737168 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.737169 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.737318 (+ 149us) server_negotiation.cc:407] Connection header received
0807 00:40:00.737382 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.737383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.737394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.737455 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:00.737621 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.738100 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.738888 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.739079 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.739159 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.739320 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.739359 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.739417 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.739472 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.739495 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:40:00.739594 4949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.737016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37487 (local address 127.1.116.131:44311)
0807 00:40:00.737241 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.737252 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.737328 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.737471 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.737471 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.737473 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:40:00.737557 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.737558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.738178 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.738179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.738807 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.738809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.739162 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.739164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.739224 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.739237 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.739419 (+ 182us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.739421 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.739469 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.739509 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":93,"threads_started":1}
I20260807 00:40:00.741688 4539 raft_consensus.cc:1261] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Refusing update from remote peer 6c794029feaa4e04b5b99f178a325e79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:00.741827 4323 raft_consensus.cc:1261] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Refusing update from remote peer 6c794029feaa4e04b5b99f178a325e79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:00.741952 4938 consensus_queue.cc:1035] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:00.742141 4938 consensus_queue.cc:1035] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:00.750303 4863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.747565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:35485 (local address 127.1.116.131:32803)
0807 00:40:00.747605 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:00.747607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:00.747615 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:40:00.747657 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:00.747658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:00.747668 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:00.747715 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:00.747890 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.748624 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.749609 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.749829 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.749880 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.750099 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.750125 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.750168 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:00.750244 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 00:40:00.750260 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:40:00.750398 4922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:00.747510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:32803 (local address 127.1.116.129:35485)
0807 00:40:00.747560 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:00.747569 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:00.747605 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:00.747757 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:00.747757 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:00.747759 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:40:00.747793 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.747794 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.748719 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.748720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.749502 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:00.749504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:00.749919 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:00.749920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:00.750001 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:00.750013 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:40:00.750177 (+ 164us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:40:00.750179 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:40:00.750241 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:40:00.750309 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
W20260807 00:40:00.750756 4911 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 00:40:02.284677 4095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:02.284825 4095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53332:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:02.285925 4095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Sending DeleteTablet for 3 replicas of tablet e78b31d983e549e5b3b282077d5d0d0c
I20260807 00:40:02.286291 4727 tablet_service.cc:1514] Processing DeleteTablet for tablet e78b31d983e549e5b3b282077d5d0d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:02 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:51162
I20260807 00:40:02.286309 4511 tablet_service.cc:1514] Processing DeleteTablet for tablet e78b31d983e549e5b3b282077d5d0d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:02 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:36418
I20260807 00:40:02.286391 4296 tablet_service.cc:1514] Processing DeleteTablet for tablet e78b31d983e549e5b3b282077d5d0d0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:02 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:48750
I20260807 00:40:02.286644 4996 tablet_replica.cc:330] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: stopping tablet replica
I20260807 00:40:02.286723 4996 raft_consensus.cc:2229] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:02.286873 4996 raft_consensus.cc:2258] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:02.286883 4998 tablet_replica.cc:330] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: stopping tablet replica
I20260807 00:40:02.286914 4997 tablet_replica.cc:330] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: stopping tablet replica
I20260807 00:40:02.287009 4997 raft_consensus.cc:2229] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:02.287092 4997 raft_consensus.cc:2258] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:02.287149 4998 raft_consensus.cc:2229] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:02.287269 4996 ts_tablet_manager.cc:1905] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:02.287290 4998 raft_consensus.cc:2258] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:02.287434 4997 ts_tablet_manager.cc:1905] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:02.287603 4998 ts_tablet_manager.cc:1905] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:02.289513 4997 ts_tablet_manager.cc:1918] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:40:02.289665 4997 log.cc:1196] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/e78b31d983e549e5b3b282077d5d0d0c
I20260807 00:40:02.289655 4996 ts_tablet_manager.cc:1918] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:40:02.289711 4996 log.cc:1196] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/e78b31d983e549e5b3b282077d5d0d0c
I20260807 00:40:02.289746 4998 ts_tablet_manager.cc:1918] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:40:02.289857 4998 log.cc:1196] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/e78b31d983e549e5b3b282077d5d0d0c
I20260807 00:40:02.290033 4996 ts_tablet_manager.cc:1939] T e78b31d983e549e5b3b282077d5d0d0c P 6c794029feaa4e04b5b99f178a325e79: Deleting consensus metadata
I20260807 00:40:02.290124 4997 ts_tablet_manager.cc:1939] T e78b31d983e549e5b3b282077d5d0d0c P e8b670e43eb146d3a676debb1e881a86: Deleting consensus metadata
I20260807 00:40:02.290179 4998 ts_tablet_manager.cc:1939] T e78b31d983e549e5b3b282077d5d0d0c P eb00dea7540e47178d64e5236cdc1302: Deleting consensus metadata
I20260807 00:40:02.290465 4070 catalog_manager.cc:4841] TS 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803): tablet e78b31d983e549e5b3b282077d5d0d0c (table test-table [id=708b072a1963481e892e5e5a1370bdfc]) successfully deleted
I20260807 00:40:02.290582 4069 catalog_manager.cc:4841] TS e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821): tablet e78b31d983e549e5b3b282077d5d0d0c (table test-table [id=708b072a1963481e892e5e5a1370bdfc]) successfully deleted
I20260807 00:40:02.290591 4070 catalog_manager.cc:4841] TS eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487): tablet e78b31d983e549e5b3b282077d5d0d0c (table test-table [id=708b072a1963481e892e5e5a1370bdfc]) successfully deleted
Aug 07 00:40:02 dist-test-slave-h6n0 krb5kdc[4030](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063202, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:40:02 dist-test-slave-h6n0 krb5kdc[4030](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063202, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:02.315514 5009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:02.307822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53348 (local address 127.1.116.190:43447)
0807 00:40:02.308046 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:02.308049 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:02.308062 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:02.308100 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:02.308100 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:02.308112 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:02.308187 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:02.308797 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:02.309302 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:02.310101 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:02.310289 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:02.310343 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:02.312796 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:02.312797 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:02.312797 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:02.312804 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:02.314325 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:02.314792 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:02.314793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:02.314793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:02.314850 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:02.315080 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:02.315080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:02.315080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:02.315177 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:02.315253 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:02.315375 (+ 122us) server_negotiation.cc:299] Negotiation successful
0807 00:40:02.315438 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":127,"threads_started":1}
I20260807 00:40:02.317276 4095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:02.317490 4095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:02.319280 4511 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0358a63df2433f9be65c5c2bbcb135 (DEFAULT_TABLE table=test-table [id=57f9882597544158b7c0f43f525dd93e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:02.319320 4296 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0358a63df2433f9be65c5c2bbcb135 (DEFAULT_TABLE table=test-table [id=57f9882597544158b7c0f43f525dd93e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:02.319341 4727 tablet_service.cc:1467] Processing CreateTablet for tablet 0d0358a63df2433f9be65c5c2bbcb135 (DEFAULT_TABLE table=test-table [id=57f9882597544158b7c0f43f525dd93e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:02.319435 4511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0358a63df2433f9be65c5c2bbcb135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:02.319435 4296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0358a63df2433f9be65c5c2bbcb135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:02.319532 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0d0358a63df2433f9be65c5c2bbcb135. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:02.320873 5010 tablet_bootstrap.cc:492] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: Bootstrap starting.
I20260807 00:40:02.321316 5010 tablet_bootstrap.cc:654] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:02.321331 5013 tablet_bootstrap.cc:492] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: Bootstrap starting.
I20260807 00:40:02.321849 5013 tablet_bootstrap.cc:654] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:02.322171 5010 tablet_bootstrap.cc:492] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: No bootstrap required, opened a new log
I20260807 00:40:02.322244 5010 ts_tablet_manager.cc:1397] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:02.322355 5012 tablet_bootstrap.cc:492] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: Bootstrap starting.
I20260807 00:40:02.322397 5010 raft_consensus.cc:348] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.322479 5010 raft_consensus.cc:374] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:02.322515 5010 raft_consensus.cc:729] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c794029feaa4e04b5b99f178a325e79, State: Initialized, Role: FOLLOWER
I20260807 00:40:02.322647 5010 consensus_queue.cc:260] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.322824 5010 ts_tablet_manager.cc:1428] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:02.322862 5012 tablet_bootstrap.cc:654] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:02.323542 5013 tablet_bootstrap.cc:492] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: No bootstrap required, opened a new log
I20260807 00:40:02.323617 5013 ts_tablet_manager.cc:1397] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:02.323639 5012 tablet_bootstrap.cc:492] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: No bootstrap required, opened a new log
I20260807 00:40:02.323696 5012 ts_tablet_manager.cc:1397] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:02.323770 5013 raft_consensus.cc:348] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.323858 5013 raft_consensus.cc:374] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:02.323908 5013 raft_consensus.cc:729] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Initialized, Role: FOLLOWER
I20260807 00:40:02.323912 5012 raft_consensus.cc:348] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.323966 5013 consensus_queue.cc:260] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.323993 5012 raft_consensus.cc:374] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:02.324026 5012 raft_consensus.cc:729] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b670e43eb146d3a676debb1e881a86, State: Initialized, Role: FOLLOWER
I20260807 00:40:02.324085 5013 ts_tablet_manager.cc:1428] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:02.324092 5012 consensus_queue.cc:260] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.324270 5012 ts_tablet_manager.cc:1428] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:02.439045 4979 raft_consensus.cc:482] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:02.439159 4979 raft_consensus.cc:504] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.439363 4979 leader_election.cc:288] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487), e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821)
I20260807 00:40:02.439661 4323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0358a63df2433f9be65c5c2bbcb135" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb00dea7540e47178d64e5236cdc1302" is_pre_election: true
I20260807 00:40:02.439661 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0358a63df2433f9be65c5c2bbcb135" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86" is_pre_election: true
I20260807 00:40:02.439771 4539 raft_consensus.cc:2454] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 0.
I20260807 00:40:02.439772 4323 raft_consensus.cc:2454] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 0.
I20260807 00:40:02.439983 4657 leader_election.cc:302] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c794029feaa4e04b5b99f178a325e79, e8b670e43eb146d3a676debb1e881a86; no voters:
I20260807 00:40:02.440191 4979 raft_consensus.cc:2781] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:02.440238 4979 raft_consensus.cc:482] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:02.440254 4979 raft_consensus.cc:3037] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:02.441160 4979 raft_consensus.cc:504] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.441341 4979 leader_election.cc:288] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 election: Requested vote from peers eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487), e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821)
I20260807 00:40:02.441582 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0358a63df2433f9be65c5c2bbcb135" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86"
I20260807 00:40:02.441540 4323 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0d0358a63df2433f9be65c5c2bbcb135" candidate_uuid: "6c794029feaa4e04b5b99f178a325e79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb00dea7540e47178d64e5236cdc1302"
I20260807 00:40:02.441670 4539 raft_consensus.cc:3037] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:02.441674 4323 raft_consensus.cc:3037] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:02.442351 4539 raft_consensus.cc:2454] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 1.
I20260807 00:40:02.442432 4323 raft_consensus.cc:2454] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c794029feaa4e04b5b99f178a325e79 in term 1.
I20260807 00:40:02.442593 4657 leader_election.cc:302] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c794029feaa4e04b5b99f178a325e79, e8b670e43eb146d3a676debb1e881a86; no voters:
I20260807 00:40:02.442736 4979 raft_consensus.cc:2781] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:02.442809 4979 raft_consensus.cc:686] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 1 LEADER]: Becoming Leader. State: Replica: 6c794029feaa4e04b5b99f178a325e79, State: Running, Role: LEADER
I20260807 00:40:02.442871 4979 consensus_queue.cc:237] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.443667 4095 catalog_manager.cc:5495] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131). New cstate: current_term: 1 leader_uuid: "6c794029feaa4e04b5b99f178a325e79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } health_report { overall_health: HEALTHY } } }
I20260807 00:40:02.473443 5030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:02.470709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.1.116.131:32803)
0807 00:40:02.470919 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:02.470921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:02.470930 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:02.470965 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:02.470966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:02.470976 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:02.471058 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:02.471264 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:02.471942 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:02.472734 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:02.472948 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:02.473021 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:02.473168 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:02.473201 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:02.473257 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:02.473331 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 00:40:02.473350 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":107,"threads_started":1}
I20260807 00:40:02.476243 4323 raft_consensus.cc:1261] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Refusing update from remote peer 6c794029feaa4e04b5b99f178a325e79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:02.476234 4539 raft_consensus.cc:1261] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Refusing update from remote peer 6c794029feaa4e04b5b99f178a325e79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:02.476481 4979 consensus_queue.cc:1035] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:02.476634 4989 consensus_queue.cc:1035] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:02.488098 4095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:02.488190 4095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:02.488862 4095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Sending DeleteTablet for 3 replicas of tablet 0d0358a63df2433f9be65c5c2bbcb135
I20260807 00:40:02.489147 4727 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0358a63df2433f9be65c5c2bbcb135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:02 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:51162
I20260807 00:40:02.489137 4511 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0358a63df2433f9be65c5c2bbcb135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:02 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:36418
I20260807 00:40:02.489204 4296 tablet_service.cc:1514] Processing DeleteTablet for tablet 0d0358a63df2433f9be65c5c2bbcb135 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:02 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:48750
I20260807 00:40:02.489246 4996 tablet_replica.cc:330] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: stopping tablet replica
I20260807 00:40:02.489264 4997 tablet_replica.cc:330] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: stopping tablet replica
I20260807 00:40:02.489305 4996 raft_consensus.cc:2229] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:02.489318 4997 raft_consensus.cc:2229] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:02.489367 4997 raft_consensus.cc:2258] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:02.489369 4996 raft_consensus.cc:2258] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:02.489452 4998 tablet_replica.cc:330] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: stopping tablet replica
I20260807 00:40:02.489419 4095 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 00:40:02.489523 4998 raft_consensus.cc:2229] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:02.489562 4998 raft_consensus.cc:2258] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 00:40:02.489563 4095 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:02.489919 4996 ts_tablet_manager.cc:1905] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:02.490052 4997 ts_tablet_manager.cc:1905] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:02.491559 4996 ts_tablet_manager.cc:1918] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:02.491590 4997 ts_tablet_manager.cc:1918] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:02.491616 4996 log.cc:1196] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/0d0358a63df2433f9be65c5c2bbcb135
I20260807 00:40:02.491607 4998 ts_tablet_manager.cc:1905] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:02.491641 4997 log.cc:1196] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/0d0358a63df2433f9be65c5c2bbcb135
I20260807 00:40:02.491873 4997 ts_tablet_manager.cc:1939] T 0d0358a63df2433f9be65c5c2bbcb135 P e8b670e43eb146d3a676debb1e881a86: Deleting consensus metadata
I20260807 00:40:02.491892 4996 ts_tablet_manager.cc:1939] T 0d0358a63df2433f9be65c5c2bbcb135 P 6c794029feaa4e04b5b99f178a325e79: Deleting consensus metadata
I20260807 00:40:02.492285 4070 catalog_manager.cc:4841] TS 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803): tablet 0d0358a63df2433f9be65c5c2bbcb135 (table test-table [id=57f9882597544158b7c0f43f525dd93e]) successfully deleted
I20260807 00:40:02.492565 4069 catalog_manager.cc:4841] TS e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821): tablet 0d0358a63df2433f9be65c5c2bbcb135 (table test-table [id=57f9882597544158b7c0f43f525dd93e]) successfully deleted
I20260807 00:40:02.492918 4511 tablet_service.cc:1467] Processing CreateTablet for tablet a95997664fc946e5b41077342c984731 (DEFAULT_TABLE table=test-table [id=d3d1932b8e0f45e0bbd09c409bb2b9aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:02.492918 4727 tablet_service.cc:1467] Processing CreateTablet for tablet a95997664fc946e5b41077342c984731 (DEFAULT_TABLE table=test-table [id=d3d1932b8e0f45e0bbd09c409bb2b9aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:02.492918 4296 tablet_service.cc:1467] Processing CreateTablet for tablet a95997664fc946e5b41077342c984731 (DEFAULT_TABLE table=test-table [id=d3d1932b8e0f45e0bbd09c409bb2b9aa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:02.493041 4296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95997664fc946e5b41077342c984731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:02.493039 4511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95997664fc946e5b41077342c984731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:02.493052 4727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a95997664fc946e5b41077342c984731. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:02.493369 4998 ts_tablet_manager.cc:1918] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:02.493418 4998 log.cc:1196] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/0d0358a63df2433f9be65c5c2bbcb135
I20260807 00:40:02.493661 4998 ts_tablet_manager.cc:1939] T 0d0358a63df2433f9be65c5c2bbcb135 P eb00dea7540e47178d64e5236cdc1302: Deleting consensus metadata
I20260807 00:40:02.493996 4070 catalog_manager.cc:4841] TS eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487): tablet 0d0358a63df2433f9be65c5c2bbcb135 (table test-table [id=57f9882597544158b7c0f43f525dd93e]) successfully deleted
I20260807 00:40:02.494343 5010 tablet_bootstrap.cc:492] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: Bootstrap starting.
I20260807 00:40:02.494364 5012 tablet_bootstrap.cc:492] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: Bootstrap starting.
I20260807 00:40:02.494632 5013 tablet_bootstrap.cc:492] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302: Bootstrap starting.
I20260807 00:40:02.494760 5010 tablet_bootstrap.cc:654] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:02.494855 5012 tablet_bootstrap.cc:654] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:02.494982 5013 tablet_bootstrap.cc:654] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:02.495479 5010 tablet_bootstrap.cc:492] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: No bootstrap required, opened a new log
I20260807 00:40:02.495522 5010 ts_tablet_manager.cc:1397] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:02.495626 5010 raft_consensus.cc:348] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.495710 5010 raft_consensus.cc:374] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:02.495748 5010 raft_consensus.cc:729] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c794029feaa4e04b5b99f178a325e79, State: Initialized, Role: FOLLOWER
I20260807 00:40:02.495808 5010 consensus_queue.cc:260] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.495860 5012 tablet_bootstrap.cc:492] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: No bootstrap required, opened a new log
I20260807 00:40:02.495914 5010 ts_tablet_manager.cc:1428] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:40:02.495914 5012 ts_tablet_manager.cc:1397] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:02.496111 5012 raft_consensus.cc:348] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.496207 5012 raft_consensus.cc:374] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:02.496243 5012 raft_consensus.cc:729] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b670e43eb146d3a676debb1e881a86, State: Initialized, Role: FOLLOWER
I20260807 00:40:02.496282 5012 consensus_queue.cc:260] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.496361 5013 tablet_bootstrap.cc:492] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302: No bootstrap required, opened a new log
I20260807 00:40:02.496392 5012 ts_tablet_manager.cc:1428] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:02.496418 5013 ts_tablet_manager.cc:1397] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:02.496558 5013 raft_consensus.cc:348] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.496676 5013 raft_consensus.cc:374] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:02.496706 5013 raft_consensus.cc:729] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Initialized, Role: FOLLOWER
I20260807 00:40:02.496759 5013 consensus_queue.cc:260] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.496878 5013 ts_tablet_manager.cc:1428] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:02.628582 4923 raft_consensus.cc:482] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:02.628701 4923 raft_consensus.cc:504] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.628881 4923 leader_election.cc:288] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821), 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803)
I20260807 00:40:02.629218 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95997664fc946e5b41077342c984731" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86" is_pre_election: true
I20260807 00:40:02.629199 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95997664fc946e5b41077342c984731" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c794029feaa4e04b5b99f178a325e79" is_pre_election: true
I20260807 00:40:02.629333 4539 raft_consensus.cc:2454] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 0.
I20260807 00:40:02.629352 4755 raft_consensus.cc:2454] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 0.
I20260807 00:40:02.629573 4226 leader_election.cc:302] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8b670e43eb146d3a676debb1e881a86, eb00dea7540e47178d64e5236cdc1302; no voters:
I20260807 00:40:02.629712 4923 raft_consensus.cc:2781] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:02.629773 4923 raft_consensus.cc:482] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:02.629796 4923 raft_consensus.cc:3037] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:02.630597 4923 raft_consensus.cc:504] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.630734 4923 leader_election.cc:288] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 election: Requested vote from peers e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821), 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803)
I20260807 00:40:02.630920 4755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95997664fc946e5b41077342c984731" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c794029feaa4e04b5b99f178a325e79"
I20260807 00:40:02.630965 4539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a95997664fc946e5b41077342c984731" candidate_uuid: "eb00dea7540e47178d64e5236cdc1302" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e8b670e43eb146d3a676debb1e881a86"
I20260807 00:40:02.631012 4755 raft_consensus.cc:3037] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:02.631029 4539 raft_consensus.cc:3037] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:02.631783 4539 raft_consensus.cc:2454] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 1.
I20260807 00:40:02.631964 4755 raft_consensus.cc:2454] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb00dea7540e47178d64e5236cdc1302 in term 1.
I20260807 00:40:02.631985 4226 leader_election.cc:302] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e8b670e43eb146d3a676debb1e881a86, eb00dea7540e47178d64e5236cdc1302; no voters:
I20260807 00:40:02.632174 4923 raft_consensus.cc:2781] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:02.632294 4923 raft_consensus.cc:686] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [term 1 LEADER]: Becoming Leader. State: Replica: eb00dea7540e47178d64e5236cdc1302, State: Running, Role: LEADER
I20260807 00:40:02.632357 4923 consensus_queue.cc:237] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } }
I20260807 00:40:02.633083 4095 catalog_manager.cc:5495] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb00dea7540e47178d64e5236cdc1302 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "eb00dea7540e47178d64e5236cdc1302" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb00dea7540e47178d64e5236cdc1302" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37487 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 } health_report { overall_health: UNKNOWN } } }
I20260807 00:40:02.643481 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:02.640765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48772 (local address 127.1.116.129:37487)
0807 00:40:02.640987 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:02.640988 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:02.640994 (+ 6us) server_negotiation.cc:407] Connection header received
0807 00:40:02.641013 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:02.641014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:02.641023 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:02.641101 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:02.641210 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:02.641759 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:02.642785 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:02.643003 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:02.643079 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:02.643224 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:02.643258 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:02.643314 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:02.643372 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 00:40:02.643393 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":144,"threads_started":1}
I20260807 00:40:02.646070 4755 raft_consensus.cc:1261] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Refusing update from remote peer eb00dea7540e47178d64e5236cdc1302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:02.646108 4539 raft_consensus.cc:1261] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Refusing update from remote peer eb00dea7540e47178d64e5236cdc1302: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:02.646373 4923 consensus_queue.cc:1035] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e8b670e43eb146d3a676debb1e881a86" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 44821 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:02.646517 4956 consensus_queue.cc:1035] T a95997664fc946e5b41077342c984731 P eb00dea7540e47178d64e5236cdc1302 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c794029feaa4e04b5b99f178a325e79" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 32803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 00:40:02.652597 4954 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 00:40:04.179713 4095 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:04.179831 4095 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53348:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:04.180855 4095 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3b797c3b8cdb4dfabeb9e08a4f4f9d8e: Sending DeleteTablet for 3 replicas of tablet a95997664fc946e5b41077342c984731
I20260807 00:40:04.181119 4296 tablet_service.cc:1514] Processing DeleteTablet for tablet a95997664fc946e5b41077342c984731 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:04 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:48750
I20260807 00:40:04.181221 4511 tablet_service.cc:1514] Processing DeleteTablet for tablet a95997664fc946e5b41077342c984731 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:04 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:36418
I20260807 00:40:04.181245 4727 tablet_service.cc:1514] Processing DeleteTablet for tablet a95997664fc946e5b41077342c984731 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:04 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:51162
I20260807 00:40:04.181408 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 4202
I20260807 00:40:04.181483 5078 tablet_replica.cc:330] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: stopping tablet replica
I20260807 00:40:04.186465 5078 raft_consensus.cc:2229] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:04.186587 5078 raft_consensus.cc:2258] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:04.189203 5078 ts_tablet_manager.cc:1905] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:04.189656 5077 tablet_replica.cc:330] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: stopping tablet replica
I20260807 00:40:04.189764 5077 raft_consensus.cc:2229] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:04.189831 5077 raft_consensus.cc:2258] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:04.190097 5077 ts_tablet_manager.cc:1905] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:04.191533 5077 ts_tablet_manager.cc:1918] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:40:04.191612 5077 log.cc:1196] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/a95997664fc946e5b41077342c984731
I20260807 00:40:04.191838 5077 ts_tablet_manager.cc:1939] T a95997664fc946e5b41077342c984731 P e8b670e43eb146d3a676debb1e881a86: Deleting consensus metadata
I20260807 00:40:04.191826 5078 ts_tablet_manager.cc:1918] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:40:04.191900 5078 log.cc:1196] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/a95997664fc946e5b41077342c984731
I20260807 00:40:04.192139 5078 ts_tablet_manager.cc:1939] T a95997664fc946e5b41077342c984731 P 6c794029feaa4e04b5b99f178a325e79: Deleting consensus metadata
I20260807 00:40:04.192320 4069 catalog_manager.cc:4841] TS e8b670e43eb146d3a676debb1e881a86 (127.1.116.130:44821): tablet a95997664fc946e5b41077342c984731 (table test-table [id=d3d1932b8e0f45e0bbd09c409bb2b9aa]) successfully deleted
I20260807 00:40:04.192548 4070 catalog_manager.cc:4841] TS 6c794029feaa4e04b5b99f178a325e79 (127.1.116.131:32803): tablet a95997664fc946e5b41077342c984731 (table test-table [id=d3d1932b8e0f45e0bbd09c409bb2b9aa]) successfully deleted
W20260807 00:40:04.197247 4070 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:37487 (error 108)
I20260807 00:40:04.197613 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 4408
I20260807 00:40:04.202127 5081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:04.197713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37487 (local address 127.0.0.1:48776)
0807 00:40:04.202015 (+ 4302us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:04.202076 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37487: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4265,"thread_start_us":99,"threads_started":1}
W20260807 00:40:04.202317 4070 catalog_manager.cc:4584] TS eb00dea7540e47178d64e5236cdc1302 (127.1.116.129:37487): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a95997664fc946e5b41077342c984731: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37487: connect: Connection refused (error 111)
I20260807 00:40:04.207193 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 4620
I20260807 00:40:04.222631 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 4049
2026-08-07T00:40:04Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6410 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:04 dist-test-slave-h6n0 krb5kdc[5085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:04 dist-test-slave-h6n0 krb5kdc[5085](info): set up 2 sockets
Aug 07 00:40:04 dist-test-slave-h6n0 krb5kdc[5085](info): commencing operation
krb5kdc: starting...
W20260807 00:40:05.479223 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:05 dist-test-slave-h6n0 krb5kdc[5085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:05.609335 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35939
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:35939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:05.701119 5102 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:05.701366 5102 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:05.701390 5102 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:05.702632 5102 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:05.702679 5102 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:05.702694 5102 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:05.702708 5102 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:05.702766 5102 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:05.704304 5102 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:35939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5102
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:05.704603 5102 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:05.704866 5102 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:05.710500 5108 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:05.710517 5112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:05.710585 5102 server_base.cc:1034] running on GCE node
W20260807 00:40:05.710505 5109 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:05.711135 5102 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:05.711444 5102 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:05.712600 5102 hybrid_clock.cc:648] HybridClock initialized: now 1786063205712579 us; error 34 us; skew 500 ppm
Aug 07 00:40:05 dist-test-slave-h6n0 krb5kdc[5085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063205, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:05.719630 5102 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:05.720101 5102 webserver.cc:460] Webserver started at http://127.1.116.190:39637/ using document root <none> and password file <none>
I20260807 00:40:05.720373 5102 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:05.720443 5102 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:05.720590 5102 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:05.721728 5102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "e2bc45f834634ab9b9170d8932867386"
format_stamp: "Formatted at 2026-08-07 00:40:05 on dist-test-slave-h6n0"
server_key: "a391074c3ef734de360f9d3fb8adc9c8"
server_key_iv: "7b98766d11f163669fa69fc7323ba7b3"
server_key_version: "encryptionkey@0"
I20260807 00:40:05.722124 5102 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "e2bc45f834634ab9b9170d8932867386"
format_stamp: "Formatted at 2026-08-07 00:40:05 on dist-test-slave-h6n0"
server_key: "a391074c3ef734de360f9d3fb8adc9c8"
server_key_iv: "7b98766d11f163669fa69fc7323ba7b3"
server_key_version: "encryptionkey@0"
I20260807 00:40:05.723587 5102 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:05.724390 5120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:05.724617 5102 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:05.724733 5102 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "e2bc45f834634ab9b9170d8932867386"
format_stamp: "Formatted at 2026-08-07 00:40:05 on dist-test-slave-h6n0"
server_key: "a391074c3ef734de360f9d3fb8adc9c8"
server_key_iv: "7b98766d11f163669fa69fc7323ba7b3"
server_key_version: "encryptionkey@0"
I20260807 00:40:05.724814 5102 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:05.755295 5102 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:05.757611 5102 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:05.757781 5102 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:05.762331 5102 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:35939
I20260807 00:40:05.762395 5207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:35939 every 8 connection(s)
I20260807 00:40:05.762782 5102 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:05.763831 5208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:05.765170 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 5102
I20260807 00:40:05.765290 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:05.765467 1490 external_mini_cluster.cc:1428] Setting key 89bb2d6614dd1ef41c25b7159287e3e2
I20260807 00:40:05.767175 5208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386: Bootstrap starting.
I20260807 00:40:05.767946 5208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:05.768332 5208 log.cc:824] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:05.769273 5208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386: No bootstrap required, opened a new log
I20260807 00:40:05.771430 5208 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bc45f834634ab9b9170d8932867386" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35939 } }
I20260807 00:40:05.771601 5208 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:05.771654 5208 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bc45f834634ab9b9170d8932867386, State: Initialized, Role: FOLLOWER
I20260807 00:40:05.771788 5208 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bc45f834634ab9b9170d8932867386" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35939 } }
I20260807 00:40:05.771884 5208 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:05.771929 5208 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:05.771976 5208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:05.772663 5208 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bc45f834634ab9b9170d8932867386" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35939 } }
I20260807 00:40:05.772782 5208 leader_election.cc:302] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2bc45f834634ab9b9170d8932867386; no voters:
I20260807 00:40:05.772976 5208 leader_election.cc:288] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [CANDIDATE]: Term 1 election: Requested vote from peers
Aug 07 00:40:05 dist-test-slave-h6n0 krb5kdc[5085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063205, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:05.773056 5215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:05.773211 5215 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [term 1 LEADER]: Becoming Leader. State: Replica: e2bc45f834634ab9b9170d8932867386, State: Running, Role: LEADER
I20260807 00:40:05.773270 5208 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:05.773386 5215 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bc45f834634ab9b9170d8932867386" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35939 } }
I20260807 00:40:05.773883 5216 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2bc45f834634ab9b9170d8932867386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bc45f834634ab9b9170d8932867386" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35939 } } }
I20260807 00:40:05.773976 5216 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:05.774189 5217 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2bc45f834634ab9b9170d8932867386. Latest consensus state: current_term: 1 leader_uuid: "e2bc45f834634ab9b9170d8932867386" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bc45f834634ab9b9170d8932867386" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35939 } } }
I20260807 00:40:05.774253 5217 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:05.774452 5228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:05.775102 5228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:05.777092 5228 catalog_manager.cc:1347] Generated new cluster ID: 287011d8d7f14522b4bffbb17fea3cc0
I20260807 00:40:05.777160 5228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:05.778371 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:05.766048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56762 (local address 127.1.116.190:35939)
0807 00:40:05.766320 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:05.766331 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:05.766373 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:40:05.767022 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:05.767033 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:05.767100 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:05.767294 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:05.767912 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:05.769300 (+ 1388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:05.770125 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:05.770642 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:05.770763 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:05.773719 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:05.773724 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:05.773725 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:05.773738 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:05.775970 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:05.776386 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:05.776389 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:05.776390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:05.776505 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:05.776741 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:05.776742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:05.776742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:05.777067 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:05.777221 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:05.777881 (+ 660us) server_negotiation.cc:299] Negotiation successful
0807 00:40:05.778060 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":123,"threads_started":1}
I20260807 00:40:05.789398 5228 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:05.790180 5228 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:05.800091 5228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2bc45f834634ab9b9170d8932867386: Generated new TSK 0
I20260807 00:40:05.800310 5228 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:40:05.829434 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35939
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:05.915285 5246 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:05.915479 5246 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:05.915503 5246 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:05.916640 5246 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:05.916687 5246 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:05.916735 5246 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:05.918190 5246 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5246
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:05.918429 5246 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:05.918694 5246 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:05.924825 5263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:05.924818 5256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:05.924800 5258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:05.925135 5246 server_base.cc:1034] running on GCE node
I20260807 00:40:05.925364 5246 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:05.925618 5246 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:05.926785 5246 hybrid_clock.cc:648] HybridClock initialized: now 1786063205926756 us; error 50 us; skew 500 ppm
Aug 07 00:40:05 dist-test-slave-h6n0 krb5kdc[5085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063205, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:05.929742 5246 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:40:05.930209 5246 webserver.cc:460] Webserver started at http://127.1.116.129:45881/ using document root <none> and password file <none>
I20260807 00:40:05.930464 5246 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:05.930531 5246 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:05.930723 5246 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:05.931728 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "4d9808fff174453b8e9ff61d55e46c4e"
format_stamp: "Formatted at 2026-08-07 00:40:05 on dist-test-slave-h6n0"
server_key: "80490cf056c461ef35c75e4bbfda5d68"
server_key_iv: "f82a6742d3270058a16f355478c1afd8"
server_key_version: "encryptionkey@0"
I20260807 00:40:05.932089 5246 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "4d9808fff174453b8e9ff61d55e46c4e"
format_stamp: "Formatted at 2026-08-07 00:40:05 on dist-test-slave-h6n0"
server_key: "80490cf056c461ef35c75e4bbfda5d68"
server_key_iv: "f82a6742d3270058a16f355478c1afd8"
server_key_version: "encryptionkey@0"
I20260807 00:40:05.933465 5246 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:05.934273 5271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:05.934650 5246 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:05.934757 5246 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "4d9808fff174453b8e9ff61d55e46c4e"
format_stamp: "Formatted at 2026-08-07 00:40:05 on dist-test-slave-h6n0"
server_key: "80490cf056c461ef35c75e4bbfda5d68"
server_key_iv: "f82a6742d3270058a16f355478c1afd8"
server_key_version: "encryptionkey@0"
I20260807 00:40:05.934839 5246 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:05.949039 5246 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:05.951308 5246 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:05.951617 5246 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:05.951985 5246 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:05.952294 5246 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:05.952363 5246 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:05.952407 5246 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:05.952438 5246 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:05.959344 5246 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:40505
I20260807 00:40:05.959412 5439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:40505 every 8 connection(s)
I20260807 00:40:05.959684 5246 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:05.964285 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 5246
I20260807 00:40:05.964372 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:05.964548 1490 external_mini_cluster.cc:1428] Setting key aa6326da7cee4bc51fed746195f07742
Aug 07 00:40:05 dist-test-slave-h6n0 krb5kdc[5085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063205, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:05.971048 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:05.960417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:33817 (local address 127.1.116.190:35939)
0807 00:40:05.960499 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:05.960501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:05.960796 (+ 295us) server_negotiation.cc:407] Connection header received
0807 00:40:05.961388 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:05.961390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:05.961401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:05.961466 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:05.962753 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:05.963276 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:05.964506 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:05.964834 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:05.964936 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:05.967641 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:05.967643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:05.967644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:05.967653 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:05.969283 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:05.969814 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:05.969814 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:05.969815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:05.969874 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:05.970195 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:05.970195 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:05.970196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:05.970317 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:05.970410 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:05.970807 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 00:40:05.970920 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:40:05.971262 5442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:05.960383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35939 (local address 127.1.116.129:33817)
0807 00:40:05.960584 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:05.960663 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:05.961279 (+ 616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:05.961551 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:05.961555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:05.962304 (+ 749us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:05.962654 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:05.962661 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:05.963367 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:05.963368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:05.964407 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:05.964411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:05.964947 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:05.964948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:05.965100 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:05.965742 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:05.965752 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:05.967525 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:05.969390 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:05.969397 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:05.969399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:05.969727 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:05.969951 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:05.969952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:05.969953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:05.970111 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:05.970406 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:05.970416 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:05.970620 (+ 204us) client_negotiation.cc:769] Sending connection context
0807 00:40:05.970779 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 00:40:05.970974 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260807 00:40:05.971868 5440 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35939
I20260807 00:40:05.971976 5440 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:05.972157 5440 heartbeater.cc:511] Master 127.1.116.190:35939 requested a full tablet report, sending...
I20260807 00:40:05.972857 5146 ts_manager.cc:194] Registered new tserver with Master: 4d9808fff174453b8e9ff61d55e46c4e (127.1.116.129:40505)
I20260807 00:40:05.973974 5146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:33817
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:40:05.993294 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35939
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:06.078265 5445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:06.078478 5445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:06.078516 5445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:06.079692 5445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:06.079762 5445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:06.079809 5445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:06.081254 5445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:06.081507 5445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:06.081727 5445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:06.086704 5459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:06.086748 5455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:06.087052 5445 server_base.cc:1034] running on GCE node
W20260807 00:40:06.086730 5452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:06.087405 5445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:06.087785 5445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:06.088990 5445 hybrid_clock.cc:648] HybridClock initialized: now 1786063206088955 us; error 48 us; skew 500 ppm
Aug 07 00:40:06 dist-test-slave-h6n0 krb5kdc[5085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:06.092068 5445 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:40:06.092561 5445 webserver.cc:460] Webserver started at http://127.1.116.130:33675/ using document root <none> and password file <none>
I20260807 00:40:06.092813 5445 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:06.092882 5445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:06.093027 5445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:06.094197 5445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "e46e9c6e524b41ad9e0ad5439e491435"
format_stamp: "Formatted at 2026-08-07 00:40:06 on dist-test-slave-h6n0"
server_key: "2a428e5ad938ad6e992f8497d8785470"
server_key_iv: "060e92550c7e02a19154c0204b092469"
server_key_version: "encryptionkey@0"
I20260807 00:40:06.094647 5445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "e46e9c6e524b41ad9e0ad5439e491435"
format_stamp: "Formatted at 2026-08-07 00:40:06 on dist-test-slave-h6n0"
server_key: "2a428e5ad938ad6e992f8497d8785470"
server_key_iv: "060e92550c7e02a19154c0204b092469"
server_key_version: "encryptionkey@0"
I20260807 00:40:06.096190 5445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:06.097157 5468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:06.097426 5445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:06.097551 5445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "e46e9c6e524b41ad9e0ad5439e491435"
format_stamp: "Formatted at 2026-08-07 00:40:06 on dist-test-slave-h6n0"
server_key: "2a428e5ad938ad6e992f8497d8785470"
server_key_iv: "060e92550c7e02a19154c0204b092469"
server_key_version: "encryptionkey@0"
I20260807 00:40:06.097649 5445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:06.108476 5445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:06.110682 5445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:06.110842 5445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:06.111104 5445 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:06.111459 5445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:06.111516 5445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:06.111553 5445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:06.111598 5445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:06.119027 5445 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:42119
I20260807 00:40:06.119076 5647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:42119 every 8 connection(s)
I20260807 00:40:06.119429 5445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
Aug 07 00:40:06 dist-test-slave-h6n0 krb5kdc[5085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:06.128963 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:06.120128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:55777 (local address 127.1.116.190:35939)
0807 00:40:06.120248 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:06.120251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:06.120545 (+ 294us) server_negotiation.cc:407] Connection header received
0807 00:40:06.121065 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:06.121066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:06.121077 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:06.121194 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:06.122175 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.122808 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.123806 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.124002 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.124063 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:06.126118 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:06.126119 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:06.126120 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:06.126127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:06.127616 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:06.128040 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:06.128041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:06.128041 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:06.128106 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:06.128395 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:06.128395 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:06.128396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:06.128513 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:06.128573 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:06.128804 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 00:40:06.128889 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 00:40:06.129091 5652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:06.120123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35939 (local address 127.1.116.130:55777)
0807 00:40:06.120368 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:06.120428 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:06.120962 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:06.121276 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:06.121279 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:06.121857 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:06.122058 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:06.122063 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.122908 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.122909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:06.123719 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:06.123721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.124085 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.124086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:06.124188 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:06.124682 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:06.124691 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:06.126024 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:06.127697 (+ 1673us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:06.127703 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:06.127705 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:06.127968 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:06.128176 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:06.128177 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:06.128177 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:06.128329 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:06.128577 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:06.128583 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:06.128689 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 00:40:06.128774 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 00:40:06.128902 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":127,"threads_started":1}
I20260807 00:40:06.129446 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 5445
I20260807 00:40:06.129547 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:06.129738 1490 external_mini_cluster.cc:1428] Setting key 0068a470f3128744b305aebdf2527e5a
I20260807 00:40:06.129928 5650 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35939
I20260807 00:40:06.130046 5650 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:06.130250 5650 heartbeater.cc:511] Master 127.1.116.190:35939 requested a full tablet report, sending...
I20260807 00:40:06.130676 5146 ts_manager.cc:194] Registered new tserver with Master: e46e9c6e524b41ad9e0ad5439e491435 (127.1.116.130:42119)
I20260807 00:40:06.131088 5146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:55777
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:40:06.157840 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35939
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:46723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:06.248519 5658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:06.248750 5658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:06.248775 5658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:06.249899 5658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:06.249934 5658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:06.249960 5658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:06.251423 5658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:46723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:06.251631 5658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:06.251832 5658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:06.257359 5669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:06.257496 5658 server_base.cc:1034] running on GCE node
W20260807 00:40:06.257357 5666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:06.257402 5667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:06.257933 5658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:06.258244 5658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:06.259438 5658 hybrid_clock.cc:648] HybridClock initialized: now 1786063206259415 us; error 48 us; skew 500 ppm
Aug 07 00:40:06 dist-test-slave-h6n0 krb5kdc[5085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:06.262291 5658 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:40:06.262874 5658 webserver.cc:460] Webserver started at http://127.1.116.131:35729/ using document root <none> and password file <none>
I20260807 00:40:06.263123 5658 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:06.263206 5658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:06.263360 5658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:06.264537 5658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "a2aad0affd34491cb5406351fe734150"
format_stamp: "Formatted at 2026-08-07 00:40:06 on dist-test-slave-h6n0"
server_key: "714291c9972c6345cef061b08e0491ec"
server_key_iv: "dcdbbb12cd8bd38761cf7986155ed941"
server_key_version: "encryptionkey@0"
I20260807 00:40:06.264948 5658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "a2aad0affd34491cb5406351fe734150"
format_stamp: "Formatted at 2026-08-07 00:40:06 on dist-test-slave-h6n0"
server_key: "714291c9972c6345cef061b08e0491ec"
server_key_iv: "dcdbbb12cd8bd38761cf7986155ed941"
server_key_version: "encryptionkey@0"
I20260807 00:40:06.266419 5658 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:06.267324 5681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:06.267556 5658 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:06.267691 5658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "a2aad0affd34491cb5406351fe734150"
format_stamp: "Formatted at 2026-08-07 00:40:06 on dist-test-slave-h6n0"
server_key: "714291c9972c6345cef061b08e0491ec"
server_key_iv: "dcdbbb12cd8bd38761cf7986155ed941"
server_key_version: "encryptionkey@0"
I20260807 00:40:06.267771 5658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:06.278829 5658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:06.281225 5658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:06.281389 5658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:06.281643 5658 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:06.281988 5658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:06.282042 5658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:06.282084 5658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:06.282119 5658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:06.289569 5658 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:46577
I20260807 00:40:06.289621 5857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:46577 every 8 connection(s)
I20260807 00:40:06.289971 5658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:06.293828 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 5658
I20260807 00:40:06.293922 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:06.294127 1490 external_mini_cluster.cc:1428] Setting key 5b68bbe3bd06496fe4da4b9aa42ebbc6
Aug 07 00:40:06 dist-test-slave-h6n0 krb5kdc[5085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063206, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:06.300391 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:06.290796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:52049 (local address 127.1.116.190:35939)
0807 00:40:06.290890 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:06.290894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:06.291115 (+ 221us) server_negotiation.cc:407] Connection header received
0807 00:40:06.291651 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:06.291653 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:06.291668 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:06.291735 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:06.292707 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.293253 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.294507 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.294862 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.294943 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:06.297177 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:06.297179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:06.297179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:06.297187 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:06.298701 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:06.299159 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:06.299160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:06.299161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:06.299244 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:06.299518 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:06.299519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:06.299519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:06.299656 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:06.299723 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:06.300185 (+ 462us) server_negotiation.cc:299] Negotiation successful
0807 00:40:06.300304 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:40:06.300478 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:06.290787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35939 (local address 127.1.116.131:52049)
0807 00:40:06.290990 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:06.291036 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:06.291548 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:06.291791 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:06.291795 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:06.292407 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:06.292613 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:06.292617 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.293370 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.293371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:06.294389 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:06.294391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:06.294962 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:06.294964 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:06.295120 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:06.295685 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:06.295694 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:06.297081 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:06.298789 (+ 1708us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:06.298796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:06.298798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:06.299034 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:06.299326 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:06.299327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:06.299327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:06.299431 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:06.299726 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:06.299730 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:06.299928 (+ 198us) client_negotiation.cc:769] Sending connection context
0807 00:40:06.300030 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 00:40:06.300213 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":121,"threads_started":1}
I20260807 00:40:06.301108 5858 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35939
I20260807 00:40:06.301199 5858 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:06.301409 5858 heartbeater.cc:511] Master 127.1.116.190:35939 requested a full tablet report, sending...
I20260807 00:40:06.301934 5146 ts_manager.cc:194] Registered new tserver with Master: a2aad0affd34491cb5406351fe734150 (127.1.116.131:46577)
I20260807 00:40:06.302343 5146 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:52049
I20260807 00:40:06.306277 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 00:40:06.321015 5211 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:06.319981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56770 (local address 127.1.116.190:35939)
0807 00:40:06.320070 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:06.320072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:06.320205 (+ 133us) server_negotiation.cc:407] Connection header received
0807 00:40:06.320232 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:06.320232 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:06.320243 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:06.320293 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:06.320920 (+ 627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56770: BlockingRecv error: recv got EOF from 127.0.0.1:56770 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:40:06.321805 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 5246
I20260807 00:40:06.332742 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 5445
I20260807 00:40:06.353394 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 5658
I20260807 00:40:06.368484 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 5102
2026-08-07T00:40:06Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2137 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:06 dist-test-slave-h6n0 krb5kdc[5877](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:06 dist-test-slave-h6n0 krb5kdc[5877](info): set up 2 sockets
Aug 07 00:40:06 dist-test-slave-h6n0 krb5kdc[5877](info): commencing operation
krb5kdc: starting...
W20260807 00:40:07.612604 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:07 dist-test-slave-h6n0 krb5kdc[5877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:07.741493 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35315
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43455
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:35315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:07.836048 5898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:07.836489 5898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:07.836534 5898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:07.837849 5898 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:07.837927 5898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:07.837952 5898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:07.837982 5898 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:07.838020 5898 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:07.839762 5898 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43455
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:35315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:07.840077 5898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:07.840345 5898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:07.845763 5908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:07.845841 5909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:07.845763 5911 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:07.846119 5898 server_base.cc:1034] running on GCE node
I20260807 00:40:07.846369 5898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:07.846743 5898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:07.847934 5898 hybrid_clock.cc:648] HybridClock initialized: now 1786063207847885 us; error 86 us; skew 500 ppm
Aug 07 00:40:07 dist-test-slave-h6n0 krb5kdc[5877](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063207, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:07.856482 5898 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:07.856967 5898 webserver.cc:460] Webserver started at http://127.1.116.190:39687/ using document root <none> and password file <none>
I20260807 00:40:07.857216 5898 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:07.857264 5898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:07.857362 5898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:07.858436 5898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "2ed16106b62a4ad4b56e41abfda94cac"
format_stamp: "Formatted at 2026-08-07 00:40:07 on dist-test-slave-h6n0"
server_key: "03269667e25ccb2d6d04cc85a19ff2f2"
server_key_iv: "d8342a51da9d0d58c7ab547b6c1a0d39"
server_key_version: "encryptionkey@0"
I20260807 00:40:07.858850 5898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "2ed16106b62a4ad4b56e41abfda94cac"
format_stamp: "Formatted at 2026-08-07 00:40:07 on dist-test-slave-h6n0"
server_key: "03269667e25ccb2d6d04cc85a19ff2f2"
server_key_iv: "d8342a51da9d0d58c7ab547b6c1a0d39"
server_key_version: "encryptionkey@0"
I20260807 00:40:07.860281 5898 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:07.861182 5923 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:07.861472 5898 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:07.861588 5898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "2ed16106b62a4ad4b56e41abfda94cac"
format_stamp: "Formatted at 2026-08-07 00:40:07 on dist-test-slave-h6n0"
server_key: "03269667e25ccb2d6d04cc85a19ff2f2"
server_key_iv: "d8342a51da9d0d58c7ab547b6c1a0d39"
server_key_version: "encryptionkey@0"
I20260807 00:40:07.861680 5898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:07.877293 5898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:07.880306 5898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:07.880504 5898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:07.885350 5898 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:35315
I20260807 00:40:07.885423 6005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:35315 every 8 connection(s)
I20260807 00:40:07.885820 5898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:07.886961 6007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:07.887524 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 5898
I20260807 00:40:07.887655 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:07.887863 1490 external_mini_cluster.cc:1428] Setting key 290cbc4dc876e107472ee6af8bb5d8d8
I20260807 00:40:07.890656 6007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac: Bootstrap starting.
I20260807 00:40:07.891654 6007 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:07.892027 6007 log.cc:824] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:07.892858 6007 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac: No bootstrap required, opened a new log
Aug 07 00:40:07 dist-test-slave-h6n0 krb5kdc[5877](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:07.895083 6007 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed16106b62a4ad4b56e41abfda94cac" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35315 } }
I20260807 00:40:07.895320 6007 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:07.895391 6007 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed16106b62a4ad4b56e41abfda94cac, State: Initialized, Role: FOLLOWER
I20260807 00:40:07.895550 6007 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed16106b62a4ad4b56e41abfda94cac" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35315 } }
I20260807 00:40:07.895640 6007 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:07.895689 6007 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:07.895741 6007 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:07.896405 6007 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed16106b62a4ad4b56e41abfda94cac" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35315 } }
I20260807 00:40:07.896672 6007 leader_election.cc:302] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ed16106b62a4ad4b56e41abfda94cac; no voters:
I20260807 00:40:07.896906 6007 leader_election.cc:288] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:07.896946 6014 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:07.897092 6014 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [term 1 LEADER]: Becoming Leader. State: Replica: 2ed16106b62a4ad4b56e41abfda94cac, State: Running, Role: LEADER
I20260807 00:40:07.897189 6014 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed16106b62a4ad4b56e41abfda94cac" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35315 } }
I20260807 00:40:07.897269 6007 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:07.897562 6014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ed16106b62a4ad4b56e41abfda94cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed16106b62a4ad4b56e41abfda94cac" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35315 } } }
I20260807 00:40:07.897642 6014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:07.897802 6014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ed16106b62a4ad4b56e41abfda94cac. Latest consensus state: current_term: 1 leader_uuid: "2ed16106b62a4ad4b56e41abfda94cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ed16106b62a4ad4b56e41abfda94cac" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35315 } } }
I20260807 00:40:07.897899 6014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:07.898062 6022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:07.898970 6022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:07.899816 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:07.888471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.1.116.190:35315)
0807 00:40:07.888687 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:07.888695 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:07.888722 (+ 27us) server_negotiation.cc:407] Connection header received
0807 00:40:07.889447 (+ 725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:07.889458 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:07.889521 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:07.889672 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:07.890716 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:07.891690 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:07.892591 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:07.893064 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:07.893190 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:07.895650 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:07.895654 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:07.895654 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:07.895668 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:07.897879 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:07.898327 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:07.898330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:07.898330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:07.898435 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:07.898711 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:07.898712 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:07.898712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:07.899040 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:07.899335 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:07.899487 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 00:40:07.899638 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
I20260807 00:40:07.900846 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:40:07.901202 6022 catalog_manager.cc:1347] Generated new cluster ID: 0af818f0ceb3426d95519e770c2fdeca
I20260807 00:40:07.901255 6022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:07.909952 6022 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:07.910899 6022 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:07.916466 6022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ed16106b62a4ad4b56e41abfda94cac: Generated new TSK 0
I20260807 00:40:07.916699 6022 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:40:07.918181 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:07.913429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35654 (local address 127.1.116.190:35315)
0807 00:40:07.913478 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:07.913482 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:07.913527 (+ 45us) server_negotiation.cc:407] Connection header received
0807 00:40:07.913639 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:07.913640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:07.913653 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:07.913723 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:07.914147 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:07.915016 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:07.916400 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:07.916711 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:07.916804 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:07.917577 (+ 773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:07.917579 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:07.917579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:07.917837 (+ 258us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:07.917866 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:07.917875 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:07.917949 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:07.918050 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 00:40:07.918089 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 00:40:07.920907 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 5898
2026-08-07T00:40:07Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1557 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:07 dist-test-slave-h6n0 krb5kdc[6060](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:07 dist-test-slave-h6n0 krb5kdc[6060](info): set up 2 sockets
Aug 07 00:40:07 dist-test-slave-h6n0 krb5kdc[6060](info): commencing operation
krb5kdc: starting...
W20260807 00:40:09.206132 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.255s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:09.334846 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36091
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:09.427098 6084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:09.427332 6084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:09.427356 6084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:09.428557 6084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:09.428611 6084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:09.428627 6084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:09.428639 6084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:09.428697 6084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:09.430287 6084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:09.430610 6084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:09.430878 6084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:09.436496 6093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:09.436478 6094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:09.436478 6096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:09.436789 6084 server_base.cc:1034] running on GCE node
I20260807 00:40:09.437150 6084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:09.437506 6084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:09.438679 6084 hybrid_clock.cc:648] HybridClock initialized: now 1786063209438669 us; error 31 us; skew 500 ppm
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:09.441529 6084 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:09.442087 6084 webserver.cc:460] Webserver started at http://127.1.116.190:44031/ using document root <none> and password file <none>
I20260807 00:40:09.442348 6084 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:09.442417 6084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:09.442622 6084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:09.443876 6084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "2def97355c984068899ae20c615283eb"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "1729d037d64df75e151f37a23ab2b312"
server_key_iv: "166f4c61c13398011d07dd5eba879b6f"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.444293 6084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "2def97355c984068899ae20c615283eb"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "1729d037d64df75e151f37a23ab2b312"
server_key_iv: "166f4c61c13398011d07dd5eba879b6f"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.445870 6084 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:09.446974 6107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:09.447258 6084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:09.447392 6084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "2def97355c984068899ae20c615283eb"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "1729d037d64df75e151f37a23ab2b312"
server_key_iv: "166f4c61c13398011d07dd5eba879b6f"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.447480 6084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:09.468557 6084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:09.471215 6084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:09.471410 6084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:09.476883 6084 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:36091
I20260807 00:40:09.476951 6184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:36091 every 8 connection(s)
I20260807 00:40:09.477375 6084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:09.478747 6186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:09.479936 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 6084
I20260807 00:40:09.480029 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:09.480247 1490 external_mini_cluster.cc:1428] Setting key 3d03fa1dfc67dd743f351d8810989938
I20260807 00:40:09.481941 6186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb: Bootstrap starting.
I20260807 00:40:09.482682 6186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:09.483050 6186 log.cc:824] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:09.483855 6186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb: No bootstrap required, opened a new log
I20260807 00:40:09.485736 6186 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2def97355c984068899ae20c615283eb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36091 } }
I20260807 00:40:09.485918 6186 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:09.485973 6186 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2def97355c984068899ae20c615283eb, State: Initialized, Role: FOLLOWER
I20260807 00:40:09.486117 6186 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2def97355c984068899ae20c615283eb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36091 } }
I20260807 00:40:09.486238 6186 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:09.486287 6186 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:09.486343 6186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:09.487025 6186 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2def97355c984068899ae20c615283eb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36091 } }
I20260807 00:40:09.487159 6186 leader_election.cc:302] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2def97355c984068899ae20c615283eb; no voters:
I20260807 00:40:09.487335 6186 leader_election.cc:288] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:09.487367 6195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:09.487500 6195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [term 1 LEADER]: Becoming Leader. State: Replica: 2def97355c984068899ae20c615283eb, State: Running, Role: LEADER
I20260807 00:40:09.487638 6186 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [sys.catalog]: configured and running, proceeding with master startup.
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:09.487726 6195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2def97355c984068899ae20c615283eb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36091 } }
I20260807 00:40:09.488440 6198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2def97355c984068899ae20c615283eb. Latest consensus state: current_term: 1 leader_uuid: "2def97355c984068899ae20c615283eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2def97355c984068899ae20c615283eb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36091 } } }
I20260807 00:40:09.488567 6198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:09.488698 6196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2def97355c984068899ae20c615283eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2def97355c984068899ae20c615283eb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36091 } } }
I20260807 00:40:09.488770 6196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:09.489266 6201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:09.489974 6201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:09.491927 6201 catalog_manager.cc:1347] Generated new cluster ID: 689d5f710fea45ba9219c7b739a80c88
I20260807 00:40:09.491984 6201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:09.493666 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:09.480926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.1.116.190:36091)
0807 00:40:09.481175 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:09.481186 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:09.481228 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:40:09.481800 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:09.481809 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:09.481873 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:09.482060 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:09.482874 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.483996 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.484808 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.485270 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.485397 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:09.488392 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:09.488398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:09.488398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:09.488414 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:09.491457 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:09.491989 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:09.491992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:09.491992 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:09.492101 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:09.492312 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:09.492312 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:09.492313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:09.492633 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:09.492761 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:09.493301 (+ 540us) server_negotiation.cc:299] Negotiation successful
0807 00:40:09.493462 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":102,"threads_started":1}
I20260807 00:40:09.503641 6201 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:09.504227 6201 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:09.518680 6201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2def97355c984068899ae20c615283eb: Generated new TSK 0
I20260807 00:40:09.518929 6201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:40:09.551743 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36091
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:09.644754 6230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:09.644997 6230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:09.645023 6230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:09.646291 6230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:09.646337 6230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:09.646364 6230 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:09.647945 6230 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:09.648195 6230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:09.648406 6230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:09.654409 6239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:09.654425 6237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:09.654893 6230 server_base.cc:1034] running on GCE node
W20260807 00:40:09.654459 6242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:09.655157 6230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:09.655479 6230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:09.656697 6230 hybrid_clock.cc:648] HybridClock initialized: now 1786063209656664 us; error 42 us; skew 500 ppm
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:09.660061 6230 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:40:09.660629 6230 webserver.cc:460] Webserver started at http://127.1.116.129:36191/ using document root <none> and password file <none>
I20260807 00:40:09.660866 6230 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:09.660916 6230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:09.661024 6230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:09.662288 6230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "de7be102d0344f01925ea33b9f5b4b14"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "15ce196994dbd86f3e42ed9964086344"
server_key_iv: "bfd863c9caef347f585c1b4c910a89e5"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.662753 6230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "de7be102d0344f01925ea33b9f5b4b14"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "15ce196994dbd86f3e42ed9964086344"
server_key_iv: "bfd863c9caef347f585c1b4c910a89e5"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.664390 6230 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:09.665432 6251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:09.665685 6230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:09.665800 6230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "de7be102d0344f01925ea33b9f5b4b14"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "15ce196994dbd86f3e42ed9964086344"
server_key_iv: "bfd863c9caef347f585c1b4c910a89e5"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.665892 6230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:09.682852 6230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:09.684926 6230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:09.685041 6230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:09.685232 6230 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:09.685582 6230 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:09.685639 6230 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:09.685662 6230 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:09.685717 6230 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:09.692678 6230 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:33101
I20260807 00:40:09.692811 6422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:33101 every 8 connection(s)
I20260807 00:40:09.693104 6230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:09.698287 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 6230
I20260807 00:40:09.698415 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:09.698720 1490 external_mini_cluster.cc:1428] Setting key 3fe43343bef1f2451468c7b34e22496e
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:09.704170 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:09.693942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:38683 (local address 127.1.116.190:36091)
0807 00:40:09.694072 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:09.694082 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:09.694294 (+ 212us) server_negotiation.cc:407] Connection header received
0807 00:40:09.695029 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:09.695031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:09.695045 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:09.695127 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:09.696196 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.696748 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.697818 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.698045 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.698140 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:09.700782 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:09.700784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:09.700784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:09.700792 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:09.702487 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:09.702998 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:09.703000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:09.703000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:09.703062 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:09.703409 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:09.703411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:09.703412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:09.703563 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:09.703651 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:09.704008 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 00:40:09.704104 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 00:40:09.704494 6427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:09.693886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36091 (local address 127.1.116.129:38683)
0807 00:40:09.694127 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:09.694182 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:09.694908 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:09.695205 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:09.695210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:09.695869 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:09.696088 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:09.696093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.696852 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.696853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:09.697723 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:09.697725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.698184 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.698185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:09.698321 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:09.699037 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:09.699048 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:09.700677 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:09.702621 (+ 1944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:09.702630 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:09.702633 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:09.702902 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:09.703137 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:09.703138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:09.703138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:09.703319 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:09.703660 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:09.703669 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:09.703851 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 00:40:09.703979 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 00:40:09.704191 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260807 00:40:09.705112 6423 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36091
I20260807 00:40:09.705215 6423 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:09.705407 6423 heartbeater.cc:511] Master 127.1.116.190:36091 requested a full tablet report, sending...
I20260807 00:40:09.705982 6128 ts_manager.cc:194] Registered new tserver with Master: de7be102d0344f01925ea33b9f5b4b14 (127.1.116.129:33101)
I20260807 00:40:09.706631 6128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:38683
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:40:09.727381 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36091
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:09.821585 6431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:09.821838 6431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:09.821864 6431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:09.823141 6431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:09.823190 6431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:09.823256 6431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:09.824918 6431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6431
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:09.825139 6431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:09.825343 6431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:09.830894 6439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:09.830921 6444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:09.830905 6441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:09.831303 6431 server_base.cc:1034] running on GCE node
I20260807 00:40:09.831585 6431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:09.831856 6431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:09.833021 6431 hybrid_clock.cc:648] HybridClock initialized: now 1786063209832993 us; error 55 us; skew 500 ppm
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:09.835851 6431 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:40:09.836279 6431 webserver.cc:460] Webserver started at http://127.1.116.130:39863/ using document root <none> and password file <none>
I20260807 00:40:09.836539 6431 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:09.836611 6431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:09.836776 6431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:09.837924 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "c934c0b09cc9425ebcd5ebae5119ccfd"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "5dd4cee8cec6fe10c227e105d89fc4cd"
server_key_iv: "825ac2aa836b6804a23dab5949edb588"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.838325 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "c934c0b09cc9425ebcd5ebae5119ccfd"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "5dd4cee8cec6fe10c227e105d89fc4cd"
server_key_iv: "825ac2aa836b6804a23dab5949edb588"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.839869 6431 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:09.840899 6455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:09.841214 6431 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:09.841356 6431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "c934c0b09cc9425ebcd5ebae5119ccfd"
format_stamp: "Formatted at 2026-08-07 00:40:09 on dist-test-slave-h6n0"
server_key: "5dd4cee8cec6fe10c227e105d89fc4cd"
server_key_iv: "825ac2aa836b6804a23dab5949edb588"
server_key_version: "encryptionkey@0"
I20260807 00:40:09.841442 6431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:09.849951 6431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:09.852506 6431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:09.852661 6431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:09.852890 6431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:09.853192 6431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:09.853250 6431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:09.853293 6431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:09.853322 6431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:09.859961 6431 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:37315
I20260807 00:40:09.860021 6619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:37315 every 8 connection(s)
I20260807 00:40:09.860306 6431 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:09.863991 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 6431
I20260807 00:40:09.864087 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:09.864287 1490 external_mini_cluster.cc:1428] Setting key 77fee4c2e4ecd43ae80dcb2ff2b5eee7
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:09.871345 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:09.861229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:46499 (local address 127.1.116.190:36091)
0807 00:40:09.861274 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:09.861278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:09.861433 (+ 155us) server_negotiation.cc:407] Connection header received
0807 00:40:09.862016 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:09.862017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:09.862028 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:09.862094 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:09.863259 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.863749 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.865240 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.865541 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.865624 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:09.868068 (+ 2444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:09.868069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:09.868069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:09.868087 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:09.869437 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:09.870079 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:09.870080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:09.870080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:09.870153 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:09.870523 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:09.870525 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:09.870525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:09.870723 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:09.870806 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:09.871187 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 00:40:09.871270 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:40:09.871686 6622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:09.861093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36091 (local address 127.1.116.130:46499)
0807 00:40:09.861295 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:09.861340 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:09.861912 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:09.862218 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:09.862221 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:09.862940 (+ 719us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:09.863171 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:09.863176 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.863875 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.863876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:09.865150 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:09.865153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:09.865658 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:09.865659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:09.865780 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:09.866377 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:09.866386 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:09.867955 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:09.869552 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:09.869559 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:09.869562 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:09.869939 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:09.870248 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:09.870249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:09.870249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:09.870408 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:09.870815 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:09.870822 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:09.871011 (+ 189us) client_negotiation.cc:769] Sending connection context
0807 00:40:09.871155 (+ 144us) client_negotiation.cc:240] Negotiation successful
0807 00:40:09.871360 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":123,"threads_started":1}
I20260807 00:40:09.872340 6620 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36091
I20260807 00:40:09.872454 6620 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:09.872677 6620 heartbeater.cc:511] Master 127.1.116.190:36091 requested a full tablet report, sending...
I20260807 00:40:09.873167 6128 ts_manager.cc:194] Registered new tserver with Master: c934c0b09cc9425ebcd5ebae5119ccfd (127.1.116.130:37315)
I20260807 00:40:09.873780 6128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:46499
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:40:09.893652 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36091
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:09.986876 6627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:09.987124 6627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:09.987149 6627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:09.988456 6627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:09.988507 6627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:09.988536 6627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:09.990204 6627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:09.990484 6627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:09.990844 6627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:09.996486 6633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:09.996635 6634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:09.996500 6636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:09.996752 6627 server_base.cc:1034] running on GCE node
I20260807 00:40:09.997015 6627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:09.997229 6627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:09.998412 6627 hybrid_clock.cc:648] HybridClock initialized: now 1786063209998398 us; error 40 us; skew 500 ppm
Aug 07 00:40:09 dist-test-slave-h6n0 krb5kdc[6060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:10.001313 6627 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:40:10.001857 6627 webserver.cc:460] Webserver started at http://127.1.116.131:34803/ using document root <none> and password file <none>
I20260807 00:40:10.002104 6627 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:10.002174 6627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:10.002339 6627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:10.003585 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "8455a96335ac4b36b9c36f9685598199"
format_stamp: "Formatted at 2026-08-07 00:40:10 on dist-test-slave-h6n0"
server_key: "13321b750faad3dc6567b5de757969d9"
server_key_iv: "c187c316e003d099b72dfc1ad3f75454"
server_key_version: "encryptionkey@0"
I20260807 00:40:10.004004 6627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "8455a96335ac4b36b9c36f9685598199"
format_stamp: "Formatted at 2026-08-07 00:40:10 on dist-test-slave-h6n0"
server_key: "13321b750faad3dc6567b5de757969d9"
server_key_iv: "c187c316e003d099b72dfc1ad3f75454"
server_key_version: "encryptionkey@0"
I20260807 00:40:10.005488 6627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:10.006354 6648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:10.006611 6627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:10.006728 6627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "8455a96335ac4b36b9c36f9685598199"
format_stamp: "Formatted at 2026-08-07 00:40:10 on dist-test-slave-h6n0"
server_key: "13321b750faad3dc6567b5de757969d9"
server_key_iv: "c187c316e003d099b72dfc1ad3f75454"
server_key_version: "encryptionkey@0"
I20260807 00:40:10.006834 6627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:10.017854 6627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:10.020171 6627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:10.020340 6627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:10.020589 6627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:10.020941 6627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:10.020998 6627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:10.021041 6627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:10.021082 6627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:10.028539 6627 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:37539
I20260807 00:40:10.028592 6816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:37539 every 8 connection(s)
I20260807 00:40:10.028957 6627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:10.030282 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 6627
I20260807 00:40:10.030393 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:10.030653 1490 external_mini_cluster.cc:1428] Setting key 3918315f2580f9f64f4d9ff45f5343f3
Aug 07 00:40:10 dist-test-slave-h6n0 krb5kdc[6060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063209, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:10.039387 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:10.029781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:58095 (local address 127.1.116.190:36091)
0807 00:40:10.029858 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:10.029860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:10.030133 (+ 273us) server_negotiation.cc:407] Connection header received
0807 00:40:10.030948 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:10.030949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:10.030964 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:10.031031 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:10.032088 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:10.032636 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:10.033714 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:10.033926 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:10.034004 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:10.036333 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:10.036334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:10.036335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:10.036342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:10.037804 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:10.038259 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:10.038261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:10.038261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:10.038327 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:10.038652 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:10.038653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:10.038654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:10.038781 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:10.038857 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:10.039199 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 00:40:10.039285 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:40:10.039566 6821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:10.029738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36091 (local address 127.1.116.131:58095)
0807 00:40:10.029967 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:10.030035 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:10.030821 (+ 786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:10.031109 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:10.031114 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:10.031754 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:10.031977 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:10.031982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:10.032736 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:10.032737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:10.033600 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:10.033602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:10.034014 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:10.034015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:10.034159 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:10.034715 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:10.034724 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:10.036203 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:10.037937 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:10.037944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:10.037947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:10.038178 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:10.038397 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:10.038398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:10.038399 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:10.038507 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:10.038857 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:10.038865 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:10.039004 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 00:40:10.039112 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 00:40:10.039290 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":115,"threads_started":1}
I20260807 00:40:10.040238 6818 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36091
I20260807 00:40:10.040355 6818 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:10.040555 6818 heartbeater.cc:511] Master 127.1.116.190:36091 requested a full tablet report, sending...
I20260807 00:40:10.041013 6128 ts_manager.cc:194] Registered new tserver with Master: 8455a96335ac4b36b9c36f9685598199 (127.1.116.131:37539)
I20260807 00:40:10.041460 6128 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:58095
I20260807 00:40:10.042462 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:40:10 dist-test-slave-h6n0 krb5kdc[6060](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063210, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 00:40:10 dist-test-slave-h6n0 krb5kdc[6060](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063210, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:10.069005 6191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:10.061505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34944 (local address 127.1.116.190:36091)
0807 00:40:10.061607 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:10.061610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:10.061724 (+ 114us) server_negotiation.cc:407] Connection header received
0807 00:40:10.061818 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:10.061819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:10.061831 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:10.061908 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:10.062489 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:10.063053 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:10.063781 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:10.063984 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:10.064062 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:10.066415 (+ 2353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:10.066416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:10.066416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:10.066423 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:10.067803 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:10.068210 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:10.068211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:10.068211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:10.068262 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:10.068475 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:10.068476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:10.068477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:10.068615 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:10.068701 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:10.068862 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:40:10.068940 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260807 00:40:10.069307 6128 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34944
I20260807 00:40:10.069909 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 6230
I20260807 00:40:10.082979 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 6431
I20260807 00:40:10.094115 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 6627
I20260807 00:40:10.102564 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 6084
2026-08-07T00:40:10Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2179 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:10 dist-test-slave-h6n0 krb5kdc[6837](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:10 dist-test-slave-h6n0 krb5kdc[6837](info): set up 2 sockets
Aug 07 00:40:10 dist-test-slave-h6n0 krb5kdc[6837](info): commencing operation
krb5kdc: starting...
W20260807 00:40:11.377189 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:11 dist-test-slave-h6n0 krb5kdc[6837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:11.509439 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:45255
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:45255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:11.602303 6855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:11.602633 6855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:11.602677 6855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:11.603910 6855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:11.603986 6855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:11.604010 6855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:11.604031 6855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:11.604065 6855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:11.605787 6855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:45255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:45255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6855
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:11.606112 6855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:11.606379 6855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:11.612080 6864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:11.612069 6867 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:11.612190 6855 server_base.cc:1034] running on GCE node
W20260807 00:40:11.612069 6865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:11.612725 6855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:11.613016 6855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:11.614216 6855 hybrid_clock.cc:648] HybridClock initialized: now 1786063211614146 us; error 88 us; skew 500 ppm
Aug 07 00:40:11 dist-test-slave-h6n0 krb5kdc[6837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063211, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:11.622762 6855 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:11.623360 6855 webserver.cc:460] Webserver started at http://127.1.116.190:32905/ using document root <none> and password file <none>
I20260807 00:40:11.623639 6855 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:11.623690 6855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:11.623845 6855 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:11.625113 6855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "038047696a74435d9dd0fa6d42fe49f2"
format_stamp: "Formatted at 2026-08-07 00:40:11 on dist-test-slave-h6n0"
server_key: "db97833a63d8ab70d8801ba03c24840d"
server_key_iv: "4cf5ea3d688660cdea83f6b7de67deb3"
server_key_version: "encryptionkey@0"
I20260807 00:40:11.625531 6855 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "038047696a74435d9dd0fa6d42fe49f2"
format_stamp: "Formatted at 2026-08-07 00:40:11 on dist-test-slave-h6n0"
server_key: "db97833a63d8ab70d8801ba03c24840d"
server_key_iv: "4cf5ea3d688660cdea83f6b7de67deb3"
server_key_version: "encryptionkey@0"
I20260807 00:40:11.627269 6855 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 00:40:11.628336 6874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:11.628661 6855 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:11.628782 6855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "038047696a74435d9dd0fa6d42fe49f2"
format_stamp: "Formatted at 2026-08-07 00:40:11 on dist-test-slave-h6n0"
server_key: "db97833a63d8ab70d8801ba03c24840d"
server_key_iv: "4cf5ea3d688660cdea83f6b7de67deb3"
server_key_version: "encryptionkey@0"
I20260807 00:40:11.628893 6855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:11.654426 6855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:11.656687 6855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:11.656818 6855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:11.661624 6855 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:45255
I20260807 00:40:11.661726 6948 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:45255 every 8 connection(s)
I20260807 00:40:11.662269 6855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:11.663275 6949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:11.665722 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 6855
I20260807 00:40:11.665823 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:11.666071 1490 external_mini_cluster.cc:1428] Setting key f1bda91049f2815af2aa318a160eae27
I20260807 00:40:11.666198 6949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2: Bootstrap starting.
I20260807 00:40:11.667178 6949 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:11.667609 6949 log.cc:824] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:11.668996 6949 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2: No bootstrap required, opened a new log
I20260807 00:40:11.671806 6949 raft_consensus.cc:348] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "038047696a74435d9dd0fa6d42fe49f2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 45255 } }
I20260807 00:40:11.671995 6949 raft_consensus.cc:374] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:11.672032 6949 raft_consensus.cc:729] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 038047696a74435d9dd0fa6d42fe49f2, State: Initialized, Role: FOLLOWER
I20260807 00:40:11.672158 6949 consensus_queue.cc:260] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "038047696a74435d9dd0fa6d42fe49f2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 45255 } }
I20260807 00:40:11.672276 6949 raft_consensus.cc:388] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:11.672343 6949 raft_consensus.cc:482] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:11.672401 6949 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:11.673429 6949 raft_consensus.cc:504] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "038047696a74435d9dd0fa6d42fe49f2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 45255 } }
I20260807 00:40:11.673579 6949 leader_election.cc:302] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 038047696a74435d9dd0fa6d42fe49f2; no voters:
I20260807 00:40:11.673799 6949 leader_election.cc:288] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:11.673900 6954 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:11.674160 6954 raft_consensus.cc:686] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [term 1 LEADER]: Becoming Leader. State: Replica: 038047696a74435d9dd0fa6d42fe49f2, State: Running, Role: LEADER
I20260807 00:40:11.674163 6949 sys_catalog.cc:564] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:11.674378 6954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "038047696a74435d9dd0fa6d42fe49f2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 45255 } }
I20260807 00:40:11.674930 6956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 038047696a74435d9dd0fa6d42fe49f2. Latest consensus state: current_term: 1 leader_uuid: "038047696a74435d9dd0fa6d42fe49f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "038047696a74435d9dd0fa6d42fe49f2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 45255 } } }
I20260807 00:40:11.675031 6956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [sys.catalog]: This master's current role is: LEADER
Aug 07 00:40:11 dist-test-slave-h6n0 krb5kdc[6837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:11.675215 6955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "038047696a74435d9dd0fa6d42fe49f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "038047696a74435d9dd0fa6d42fe49f2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 45255 } } }
I20260807 00:40:11.675283 6955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:11.675421 6967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:11.676091 6967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:11.678174 6967 catalog_manager.cc:1347] Generated new cluster ID: e420010b788e487c94bf220733bc69a4
I20260807 00:40:11.678915 6967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:11.681195 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:11.666902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51640 (local address 127.1.116.190:45255)
0807 00:40:11.667183 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:11.667194 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:11.667234 (+ 40us) server_negotiation.cc:407] Connection header received
0807 00:40:11.667993 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:11.668005 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:11.668093 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:11.668456 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:11.669329 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:11.670735 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:11.671620 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:11.672240 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:11.672372 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:11.675859 (+ 3487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:11.675865 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:11.675865 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:11.675881 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:11.678611 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:11.679077 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:11.679082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:11.679083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:11.679239 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:11.679500 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:11.679501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:11.679501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:11.679897 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:11.680090 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:11.680733 (+ 643us) server_negotiation.cc:299] Negotiation successful
0807 00:40:11.680925 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":112,"threads_started":1}
I20260807 00:40:11.704147 6967 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:11.704766 6967 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:11.715615 6967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 038047696a74435d9dd0fa6d42fe49f2: Generated new TSK 0
I20260807 00:40:11.715940 6967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:40:11.754000 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:45255
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:11.840400 6991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:11.840628 6991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:11.840667 6991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:11.841769 6991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:11.841833 6991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:11.841878 6991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:11.843427 6991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:45255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:11.843672 6991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:11.843888 6991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:11.849252 6996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:11.849265 7000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:11.849298 6998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:11.849658 6991 server_base.cc:1034] running on GCE node
I20260807 00:40:11.849875 6991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:11.850103 6991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:11.851269 6991 hybrid_clock.cc:648] HybridClock initialized: now 1786063211851246 us; error 39 us; skew 500 ppm
Aug 07 00:40:11 dist-test-slave-h6n0 krb5kdc[6837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063211, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:11.853753 6991 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:40:11.854200 6991 webserver.cc:460] Webserver started at http://127.1.116.129:44287/ using document root <none> and password file <none>
I20260807 00:40:11.854430 6991 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:11.854480 6991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:11.854631 6991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:11.855628 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "f160b3f2c1e74f0ca41695af3f5779f0"
format_stamp: "Formatted at 2026-08-07 00:40:11 on dist-test-slave-h6n0"
server_key: "6d1217b3bcc0655788209fc40f2caac3"
server_key_iv: "6dad4ded1ee719cc80da6330355bee04"
server_key_version: "encryptionkey@0"
I20260807 00:40:11.855973 6991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "f160b3f2c1e74f0ca41695af3f5779f0"
format_stamp: "Formatted at 2026-08-07 00:40:11 on dist-test-slave-h6n0"
server_key: "6d1217b3bcc0655788209fc40f2caac3"
server_key_iv: "6dad4ded1ee719cc80da6330355bee04"
server_key_version: "encryptionkey@0"
I20260807 00:40:11.857334 6991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:40:11.858172 7011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:11.858443 6991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:11.858579 6991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "f160b3f2c1e74f0ca41695af3f5779f0"
format_stamp: "Formatted at 2026-08-07 00:40:11 on dist-test-slave-h6n0"
server_key: "6d1217b3bcc0655788209fc40f2caac3"
server_key_iv: "6dad4ded1ee719cc80da6330355bee04"
server_key_version: "encryptionkey@0"
I20260807 00:40:11.858656 6991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:11.878448 6991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:11.880369 6991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:11.880532 6991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:11.880755 6991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:11.881064 6991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:11.881120 6991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:11.881156 6991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:11.881183 6991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:11.887955 6991 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:33533
I20260807 00:40:11.888020 7177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:33533 every 8 connection(s)
I20260807 00:40:11.888315 6991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:11.890973 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 6991
I20260807 00:40:11.891120 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:11.891335 1490 external_mini_cluster.cc:1428] Setting key 47383d9996ea4f7da20ab5ee250680e9
Aug 07 00:40:11 dist-test-slave-h6n0 krb5kdc[6837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063211, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:11.899885 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:11.889198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:38185 (local address 127.1.116.190:45255)
0807 00:40:11.889307 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:11.889310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:11.889502 (+ 192us) server_negotiation.cc:407] Connection header received
0807 00:40:11.890672 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:11.890674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:11.890689 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:11.890782 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:11.891687 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:11.892401 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:11.894060 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:11.894303 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:11.894383 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:11.896876 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:11.896877 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:11.896878 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:11.896886 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:11.898407 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:11.898870 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:11.898871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:11.898872 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:11.898951 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:11.899230 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:11.899231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:11.899231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:11.899346 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:11.899415 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:11.899705 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 00:40:11.899791 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:40:11.900090 7180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:11.889074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:45255 (local address 127.1.116.129:38185)
0807 00:40:11.889318 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:11.889381 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:11.890501 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:11.890876 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:11.890887 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:11.891286 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:11.891569 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:11.891574 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:11.892523 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:11.892524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:11.893955 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:11.893960 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:11.894427 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:11.894428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:11.894623 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:11.895219 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:11.895228 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:11.896768 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:11.898507 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:11.898514 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:11.898516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:11.898790 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:11.899029 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:11.899030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:11.899030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:11.899150 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:11.899421 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:11.899428 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:11.899571 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 00:40:11.899670 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 00:40:11.899852 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
I20260807 00:40:11.900499 7178 heartbeater.cc:347] Connected to a master server at 127.1.116.190:45255
I20260807 00:40:11.900596 7178 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:11.900787 7178 heartbeater.cc:511] Master 127.1.116.190:45255 requested a full tablet report, sending...
I20260807 00:40:11.901347 6896 ts_manager.cc:194] Registered new tserver with Master: f160b3f2c1e74f0ca41695af3f5779f0 (127.1.116.129:33533)
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:40:11.901888 6896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:38185
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:40:11.919852 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:45255
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:12.011909 7184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:12.012179 7184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:12.012216 7184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:12.013387 7184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:12.013456 7184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:12.013504 7184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:12.015183 7184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:45255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:12.015473 7184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:12.015736 7184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:12.021500 7200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:12.021509 7195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:12.021500 7194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:12.021831 7184 server_base.cc:1034] running on GCE node
I20260807 00:40:12.022136 7184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:12.022445 7184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:12.023633 7184 hybrid_clock.cc:648] HybridClock initialized: now 1786063212023617 us; error 70 us; skew 500 ppm
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[6837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:12.026371 7184 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:40:12.026911 7184 webserver.cc:460] Webserver started at http://127.1.116.130:44065/ using document root <none> and password file <none>
I20260807 00:40:12.027150 7184 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:12.027218 7184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:12.027379 7184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:12.028527 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "13291d93d3c04311ac08ea64d2a98100"
format_stamp: "Formatted at 2026-08-07 00:40:12 on dist-test-slave-h6n0"
server_key: "06bc31aa66d93c2f7692437609aeac3f"
server_key_iv: "ecfc10d7bdda86cfa47f17e993df4f39"
server_key_version: "encryptionkey@0"
I20260807 00:40:12.028916 7184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "13291d93d3c04311ac08ea64d2a98100"
format_stamp: "Formatted at 2026-08-07 00:40:12 on dist-test-slave-h6n0"
server_key: "06bc31aa66d93c2f7692437609aeac3f"
server_key_iv: "ecfc10d7bdda86cfa47f17e993df4f39"
server_key_version: "encryptionkey@0"
I20260807 00:40:12.030437 7184 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:12.031634 7207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:12.031930 7184 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:12.032110 7184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "13291d93d3c04311ac08ea64d2a98100"
format_stamp: "Formatted at 2026-08-07 00:40:12 on dist-test-slave-h6n0"
server_key: "06bc31aa66d93c2f7692437609aeac3f"
server_key_iv: "ecfc10d7bdda86cfa47f17e993df4f39"
server_key_version: "encryptionkey@0"
I20260807 00:40:12.032197 7184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:12.061684 7184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:12.063758 7184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:12.063916 7184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:12.064152 7184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:12.064463 7184 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:12.064524 7184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:12.064567 7184 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:12.064596 7184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:12.071524 7184 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:35517
I20260807 00:40:12.071587 7375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:35517 every 8 connection(s)
I20260807 00:40:12.071877 7184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:12.076486 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 7184
I20260807 00:40:12.076630 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:12.076823 1490 external_mini_cluster.cc:1428] Setting key 2c961b804cf316055cb8695c23848615
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[6837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:12.082517 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:12.072729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:48191 (local address 127.1.116.190:45255)
0807 00:40:12.072803 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:12.072805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:12.073054 (+ 249us) server_negotiation.cc:407] Connection header received
0807 00:40:12.074094 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:12.074095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:12.074110 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:12.074186 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:12.074956 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.075527 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.076638 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.076877 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.076954 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:12.079609 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:12.079611 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:12.079611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:12.079621 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:12.081103 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.081516 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.081517 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.081517 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.081581 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.081838 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.081839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.081840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.081959 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:12.082048 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:12.082331 (+ 283us) server_negotiation.cc:299] Negotiation successful
0807 00:40:12.082425 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:40:12.082859 7379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:12.072684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:45255 (local address 127.1.116.130:48191)
0807 00:40:12.072913 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:12.072950 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:12.073952 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:12.074260 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:12.074263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:12.074497 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:12.074759 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:12.074763 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.075622 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.075622 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:12.076536 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:12.076539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.076973 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.076974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:12.077098 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:12.077738 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:12.077746 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:12.079498 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:12.081208 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:12.081215 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:12.081217 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:12.081434 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:12.081651 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:12.081651 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:12.081652 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:12.081761 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:12.082043 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:12.082049 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:12.082183 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 00:40:12.082285 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 00:40:12.082433 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":131,"threads_started":1}
I20260807 00:40:12.083315 7376 heartbeater.cc:347] Connected to a master server at 127.1.116.190:45255
I20260807 00:40:12.083412 7376 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:12.083603 7376 heartbeater.cc:511] Master 127.1.116.190:45255 requested a full tablet report, sending...
I20260807 00:40:12.083986 6896 ts_manager.cc:194] Registered new tserver with Master: 13291d93d3c04311ac08ea64d2a98100 (127.1.116.130:35517)
I20260807 00:40:12.084326 6896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:48191
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:40:12.103493 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:45255
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:38587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:12.196332 7383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:12.196568 7383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:12.196592 7383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:12.197836 7383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:12.197887 7383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:12.197914 7383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:12.199659 7383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:38587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:45255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:12.199963 7383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:12.200237 7383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:12.205996 7391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:12.206099 7383 server_base.cc:1034] running on GCE node
W20260807 00:40:12.206008 7395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:12.206008 7392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:12.206482 7383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:12.206823 7383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:12.207999 7383 hybrid_clock.cc:648] HybridClock initialized: now 1786063212207985 us; error 52 us; skew 500 ppm
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[6837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:12.210861 7383 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:40:12.211347 7383 webserver.cc:460] Webserver started at http://127.1.116.131:33653/ using document root <none> and password file <none>
I20260807 00:40:12.211591 7383 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:12.211642 7383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:12.211802 7383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:12.212934 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "a3565ad352c6454e80e6f342649aedfa"
format_stamp: "Formatted at 2026-08-07 00:40:12 on dist-test-slave-h6n0"
server_key: "492f839c3531a4ca672f4b47b12d608f"
server_key_iv: "88ad55f6f5fbe28d63134d406615a4d5"
server_key_version: "encryptionkey@0"
I20260807 00:40:12.213303 7383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "a3565ad352c6454e80e6f342649aedfa"
format_stamp: "Formatted at 2026-08-07 00:40:12 on dist-test-slave-h6n0"
server_key: "492f839c3531a4ca672f4b47b12d608f"
server_key_iv: "88ad55f6f5fbe28d63134d406615a4d5"
server_key_version: "encryptionkey@0"
I20260807 00:40:12.214855 7383 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 00:40:12.215823 7407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:12.216085 7383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:12.216207 7383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "a3565ad352c6454e80e6f342649aedfa"
format_stamp: "Formatted at 2026-08-07 00:40:12 on dist-test-slave-h6n0"
server_key: "492f839c3531a4ca672f4b47b12d608f"
server_key_iv: "88ad55f6f5fbe28d63134d406615a4d5"
server_key_version: "encryptionkey@0"
I20260807 00:40:12.216301 7383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:12.233474 7383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:12.236106 7383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:12.236265 7383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:12.236465 7383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:12.236720 7383 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:12.236754 7383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:12.236774 7383 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:12.236786 7383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:12.243371 7383 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44057
I20260807 00:40:12.243464 7569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44057 every 8 connection(s)
I20260807 00:40:12.243762 7383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:12.249666 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 7383
I20260807 00:40:12.249818 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:12.250030 1490 external_mini_cluster.cc:1428] Setting key 6305a9b61f1b8ee04d05616d9b074aa5
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[6837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:12.253767 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:12.244697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:35877 (local address 127.1.116.190:45255)
0807 00:40:12.244773 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:12.244775 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:12.245059 (+ 284us) server_negotiation.cc:407] Connection header received
0807 00:40:12.245981 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:12.245982 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:12.245995 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:12.246058 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:12.246725 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.247274 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.248300 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.248495 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.248557 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:12.251168 (+ 2611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:12.251169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:12.251169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:12.251176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:12.252512 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.252904 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.252905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.252906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.252970 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.253215 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.253216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.253216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.253338 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:12.253403 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:12.253621 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 00:40:12.253694 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:40:12.253882 7573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:12.244692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:45255 (local address 127.1.116.131:35877)
0807 00:40:12.244913 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:12.244953 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:12.245871 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:12.246140 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:12.246143 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:12.246380 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:12.246639 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:12.246644 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.247374 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.247375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:12.248216 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:12.248218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.248580 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.248581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:12.248678 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:12.249182 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:12.249192 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:12.251029 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:12.252594 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:12.252600 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:12.252602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:12.252806 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:12.253051 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:12.253051 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:12.253052 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:12.253146 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:12.253404 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:12.253408 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:12.253517 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 00:40:12.253598 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:40:12.253709 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":121,"threads_started":1}
I20260807 00:40:12.254633 7571 heartbeater.cc:347] Connected to a master server at 127.1.116.190:45255
I20260807 00:40:12.254740 7571 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:12.254946 7571 heartbeater.cc:511] Master 127.1.116.190:45255 requested a full tablet report, sending...
I20260807 00:40:12.255424 6896 ts_manager.cc:194] Registered new tserver with Master: a3565ad352c6454e80e6f342649aedfa (127.1.116.131:44057)
I20260807 00:40:12.255825 6896 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:35877
I20260807 00:40:12.262159 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[6837](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[6837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:40:12.292956 7580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:12.284812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52178 (local address 127.1.116.129:33533)
0807 00:40:12.285041 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:12.285046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:12.285055 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:12.285080 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:12.285081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:12.285095 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:12.285176 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:12.285802 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.286592 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.287331 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.287648 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.287730 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:12.290138 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:12.290147 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:12.290148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:12.290155 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:12.291737 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.292098 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.292101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.292102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.292185 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.292393 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.292394 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.292394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.292530 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:12.292680 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:12.292752 (+ 72us) server_negotiation.cc:299] Negotiation successful
0807 00:40:12.292829 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[6837](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063212, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:12.301337 6952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:12.294620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.1.116.190:45255)
0807 00:40:12.294664 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:12.294666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:12.294804 (+ 138us) server_negotiation.cc:407] Connection header received
0807 00:40:12.294863 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:12.294864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:12.294873 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:12.294913 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:12.295401 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.295881 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.296598 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:12.296772 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:12.296829 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:12.299139 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:12.299140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:12.299140 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:12.299146 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:12.300350 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.300660 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.300661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.300661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.300712 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:12.300892 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:12.300893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:12.300893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:12.301037 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:12.301115 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:12.301211 (+ 96us) server_negotiation.cc:299] Negotiation successful
0807 00:40:12.301275 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260807 00:40:12.301708 6896 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51650
I20260807 00:40:12.302168 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 6991
I20260807 00:40:12.315984 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 7184
I20260807 00:40:12.328920 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 7383
I20260807 00:40:12.347577 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 6855
2026-08-07T00:40:12Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2246 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[7589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[7589](info): set up 2 sockets
Aug 07 00:40:12 dist-test-slave-h6n0 krb5kdc[7589](info): commencing operation
krb5kdc: starting...
W20260807 00:40:13.607692 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:13 dist-test-slave-h6n0 krb5kdc[7589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:13.739545 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35073
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:35073
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:13.830717 7615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:13.830931 7615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:13.830969 7615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:13.832084 7615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:13.832154 7615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:13.832181 7615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:13.832214 7615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:13.832237 7615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:13.833740 7615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:35073
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35073
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:13.834012 7615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:13.834231 7615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:13.839751 7624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:13.839808 7621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:13.839751 7620 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:13.840070 7615 server_base.cc:1034] running on GCE node
I20260807 00:40:13.840346 7615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:13.840694 7615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:13.841878 7615 hybrid_clock.cc:648] HybridClock initialized: now 1786063213841872 us; error 36 us; skew 500 ppm
Aug 07 00:40:13 dist-test-slave-h6n0 krb5kdc[7589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063213, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:13.844815 7615 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:13.845224 7615 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:13.845276 7615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:13.845373 7615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:13.846585 7615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "6e67c5aba6314fd7bd306b5b9a057b0b"
format_stamp: "Formatted at 2026-08-07 00:40:13 on dist-test-slave-h6n0"
server_key: "ca4287dc4acf547ca026a343496275f5"
server_key_iv: "a094e86d6253ab827f3a3cc3e0ce21bf"
server_key_version: "encryptionkey@0"
I20260807 00:40:13.847004 7615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "6e67c5aba6314fd7bd306b5b9a057b0b"
format_stamp: "Formatted at 2026-08-07 00:40:13 on dist-test-slave-h6n0"
server_key: "ca4287dc4acf547ca026a343496275f5"
server_key_iv: "a094e86d6253ab827f3a3cc3e0ce21bf"
server_key_version: "encryptionkey@0"
I20260807 00:40:13.848500 7615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:40:13.849386 7631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:13.849618 7615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:40:13.849722 7615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "6e67c5aba6314fd7bd306b5b9a057b0b"
format_stamp: "Formatted at 2026-08-07 00:40:13 on dist-test-slave-h6n0"
server_key: "ca4287dc4acf547ca026a343496275f5"
server_key_iv: "a094e86d6253ab827f3a3cc3e0ce21bf"
server_key_version: "encryptionkey@0"
I20260807 00:40:13.849812 7615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:13.860744 7615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:13.862958 7615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:13.863070 7615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:13.867426 7615 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:35073
I20260807 00:40:13.867523 7704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:35073 every 8 connection(s)
I20260807 00:40:13.867803 7615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:13.869071 7706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:13.871778 7706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b: Bootstrap starting.
I20260807 00:40:13.872488 7706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:13.872836 7706 log.cc:824] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:13.873675 7706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b: No bootstrap required, opened a new log
I20260807 00:40:13.875555 7706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35073 } }
I20260807 00:40:13.875741 7706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:13.875782 7706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e67c5aba6314fd7bd306b5b9a057b0b, State: Initialized, Role: FOLLOWER
I20260807 00:40:13.875963 7706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35073 } }
I20260807 00:40:13.876051 7706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:13.876093 7706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:13.876149 7706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:13.877053 7706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35073 } }
I20260807 00:40:13.877182 7706 leader_election.cc:302] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6e67c5aba6314fd7bd306b5b9a057b0b; no voters:
I20260807 00:40:13.877398 7706 leader_election.cc:288] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:13.877522 7711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:13.877664 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 7615
I20260807 00:40:13.877729 7706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:13.877781 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:13.877784 7711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [term 1 LEADER]: Becoming Leader. State: Replica: 6e67c5aba6314fd7bd306b5b9a057b0b, State: Running, Role: LEADER
I20260807 00:40:13.877897 7711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35073 } }
I20260807 00:40:13.878008 1490 external_mini_cluster.cc:1428] Setting key e068adf660e57e568a0c896963485fdf
I20260807 00:40:13.878890 7712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35073 } } }
I20260807 00:40:13.878973 7713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6e67c5aba6314fd7bd306b5b9a057b0b. Latest consensus state: current_term: 1 leader_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6e67c5aba6314fd7bd306b5b9a057b0b" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35073 } } }
I20260807 00:40:13.879001 7712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:13.879025 7713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:13.879388 7731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:13.880462 7731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:13.882644 7731 catalog_manager.cc:1347] Generated new cluster ID: 10319ba4daa74331b283798d1ed57e4f
I20260807 00:40:13.882710 7731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:40:13 dist-test-slave-h6n0 krb5kdc[7589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:13.891481 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:13.878496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.1.116.190:35073)
0807 00:40:13.879628 (+ 1132us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:13.879639 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:13.879678 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:40:13.879820 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:13.879828 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:13.879891 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:13.880086 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:13.880866 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:13.882129 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:13.883255 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:13.883941 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:13.884049 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:13.887414 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:13.887419 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:13.887419 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:13.887433 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:13.889566 (+ 2133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:13.890091 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:13.890094 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:13.890095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:13.890206 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:13.890455 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:13.890456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:13.890457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:13.890813 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:13.890974 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:13.891103 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 00:40:13.891253 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1062,"thread_start_us":127,"threads_started":1}
I20260807 00:40:13.894886 7731 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:13.895519 7731 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:13.899684 7731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b: Generated new TSK 0
I20260807 00:40:13.899900 7731 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:40:13.931344 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35073
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:14.043524 7742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:14.043792 7742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:14.043834 7742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:14.045050 7742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:14.045115 7742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:14.045173 7742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:14.046864 7742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:14.047233 7742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:14.047535 7742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:14.053108 7752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.053157 7750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.053133 7754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:14.053514 7742 server_base.cc:1034] running on GCE node
I20260807 00:40:14.053803 7742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:14.054026 7742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:14.055199 7742 hybrid_clock.cc:648] HybridClock initialized: now 1786063214055190 us; error 31 us; skew 500 ppm
Aug 07 00:40:14 dist-test-slave-h6n0 krb5kdc[7589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063214, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:14.058099 7742 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:40:14.058511 7742 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:14.058637 7742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:14.058755 7742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:14.059880 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "50d03cd73d29410b9bea999252f5c8e3"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "6d791848700124a61990f1d716fa32df"
server_key_iv: "daf9801db7386f17ff6d4f2671f097a1"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.060225 7742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "50d03cd73d29410b9bea999252f5c8e3"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "6d791848700124a61990f1d716fa32df"
server_key_iv: "daf9801db7386f17ff6d4f2671f097a1"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.061642 7742 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:14.062419 7764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.062669 7742 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:14.062760 7742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "50d03cd73d29410b9bea999252f5c8e3"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "6d791848700124a61990f1d716fa32df"
server_key_iv: "daf9801db7386f17ff6d4f2671f097a1"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.062821 7742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:14.075337 7742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:14.077845 7742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:14.078011 7742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:14.078265 7742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:14.078792 7742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:14.078866 7742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.078899 7742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:14.078917 7742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.086863 7742 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:37365
I20260807 00:40:14.086916 7919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:37365 every 8 connection(s)
I20260807 00:40:14.087224 7742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:14.089236 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 7742
I20260807 00:40:14.089361 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:14.089593 1490 external_mini_cluster.cc:1428] Setting key 475332625a2b0e8c33badbfd3cd018f5
Aug 07 00:40:14 dist-test-slave-h6n0 krb5kdc[7589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063214, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:14.099468 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.088000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:39621 (local address 127.1.116.190:35073)
0807 00:40:14.088118 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.088121 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.088381 (+ 260us) server_negotiation.cc:407] Connection header received
0807 00:40:14.089007 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.089008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.089020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.089082 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:14.090494 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.091065 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.092743 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.092942 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.093048 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.095734 (+ 2686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:14.095735 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:14.095736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:14.095743 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:14.097605 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.098234 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.098236 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.098236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.098296 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.098672 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.098673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.098673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.098783 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:14.098921 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.099317 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.099396 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:40:14.099813 7923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.087967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35073 (local address 127.1.116.129:39621)
0807 00:40:14.088225 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.088268 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.088879 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.089187 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.089193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.089612 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:14.090323 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.090330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.091178 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.091179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.092601 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.092605 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.093091 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.093094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.093246 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.093844 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:14.093854 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:14.095624 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:14.097713 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:14.097724 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:14.097727 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:14.098106 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:14.098390 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:14.098391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:14.098392 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:14.098581 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:14.098941 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:14.098950 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:14.099155 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.099291 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.099514 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":155,"threads_started":1}
I20260807 00:40:14.100500 7921 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35073
I20260807 00:40:14.100634 7921 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:14.100843 7921 heartbeater.cc:511] Master 127.1.116.190:35073 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:40:14.101488 7656 ts_manager.cc:194] Registered new tserver with Master: 50d03cd73d29410b9bea999252f5c8e3 (127.1.116.129:37365)
I20260807 00:40:14.102101 7656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:39621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:40:14.121184 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35073
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:14.212585 7928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:14.212828 7928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:14.212852 7928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:14.214083 7928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:14.214134 7928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:14.214162 7928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:14.215885 7928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:14.216217 7928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:14.216500 7928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:14.221988 7940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.221983 7935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.221988 7937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:14.222301 7928 server_base.cc:1034] running on GCE node
I20260807 00:40:14.222647 7928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:14.222930 7928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:14.224080 7928 hybrid_clock.cc:648] HybridClock initialized: now 1786063214224064 us; error 38 us; skew 500 ppm
Aug 07 00:40:14 dist-test-slave-h6n0 krb5kdc[7589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063214, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:14.227174 7928 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:40:14.227609 7928 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:14.227666 7928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:14.227845 7928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:14.229039 7928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "c88b499aeabd4d309f338c1876ad7620"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "d74edae8a4c0a06cc609b8e72a7907ae"
server_key_iv: "2ac9faa5cd66f0ccdaebab2f2ee22bda"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.229446 7928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "c88b499aeabd4d309f338c1876ad7620"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "d74edae8a4c0a06cc609b8e72a7907ae"
server_key_iv: "2ac9faa5cd66f0ccdaebab2f2ee22bda"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.230953 7928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:14.231926 7953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.232223 7928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:14.232445 7928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "c88b499aeabd4d309f338c1876ad7620"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "d74edae8a4c0a06cc609b8e72a7907ae"
server_key_iv: "2ac9faa5cd66f0ccdaebab2f2ee22bda"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.232530 7928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:14.246855 7928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:14.249078 7928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:14.249292 7928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:14.249562 7928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:14.249930 7928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:14.249986 7928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.250051 7928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:14.250094 7928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.259689 7928 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:40927
I20260807 00:40:14.259735 8122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:40927 every 8 connection(s)
I20260807 00:40:14.260073 7928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
Aug 07 00:40:14 dist-test-slave-h6n0 krb5kdc[7589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063214, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:14.267525 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 7928
I20260807 00:40:14.267654 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:14.267946 1490 external_mini_cluster.cc:1428] Setting key fd64f0c28eea8a46ec2392cd00532d84
I20260807 00:40:14.270287 8125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.260816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35073 (local address 127.1.116.130:34471)
0807 00:40:14.261003 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.261053 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.261546 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.261840 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.261843 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.262127 (+ 284us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:14.262709 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.262714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.263486 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.263487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.264330 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.264332 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.264722 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.264723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.264818 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.265326 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:14.265334 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:14.266910 (+ 1576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:14.268702 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:14.268724 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:14.268726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:14.269005 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:14.269314 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:14.269315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:14.269315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:14.269427 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:14.269765 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:14.269771 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:14.269885 (+ 114us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.270003 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.270123 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260807 00:40:14.270310 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.260877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:34471 (local address 127.1.116.190:35073)
0807 00:40:14.260937 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.260940 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.261133 (+ 193us) server_negotiation.cc:407] Connection header received
0807 00:40:14.261662 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.261664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.261675 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.261755 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:14.262802 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.263387 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.264420 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.264635 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.264701 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.267002 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:14.267004 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:14.267005 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:14.267012 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:14.268603 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.269167 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.269168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.269169 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.269235 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.269505 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.269506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.269506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.269673 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:14.269755 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.270054 (+ 299us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.270156 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:40:14.270859 8123 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35073
I20260807 00:40:14.270994 8123 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:14.271226 8123 heartbeater.cc:511] Master 127.1.116.190:35073 requested a full tablet report, sending...
I20260807 00:40:14.271863 7656 ts_manager.cc:194] Registered new tserver with Master: c88b499aeabd4d309f338c1876ad7620 (127.1.116.130:40927)
I20260807 00:40:14.272516 7656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:34471
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:40:14.299422 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35073
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:36289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:14.392405 8129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:14.392678 8129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:14.392716 8129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:14.393901 8129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:14.393967 8129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:14.394016 8129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:14.395740 8129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35073
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:14.396027 8129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:14.396293 8129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:14.402194 8145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.402313 8139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:14.402359 8129 server_base.cc:1034] running on GCE node
W20260807 00:40:14.402319 8141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:14.402891 8129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:14.403224 8129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:14.404408 8129 hybrid_clock.cc:648] HybridClock initialized: now 1786063214404386 us; error 35 us; skew 500 ppm
Aug 07 00:40:14 dist-test-slave-h6n0 krb5kdc[7589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063214, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:14.407406 8129 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:40:14.407855 8129 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:14.407912 8129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:14.408016 8129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:14.409246 8129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "17d023f57271463b9d89aff3b4d37075"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "29cde87b98860fea932bcee6cff15814"
server_key_iv: "d186873b2a0f4437fdb6df2d2ef9a961"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.409600 8129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "17d023f57271463b9d89aff3b4d37075"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "29cde87b98860fea932bcee6cff15814"
server_key_iv: "d186873b2a0f4437fdb6df2d2ef9a961"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.411182 8129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 00:40:14.412050 8153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.412333 8129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:14.412451 8129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "17d023f57271463b9d89aff3b4d37075"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "29cde87b98860fea932bcee6cff15814"
server_key_iv: "d186873b2a0f4437fdb6df2d2ef9a961"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.412550 8129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:14.429116 8129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:14.431538 8129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:14.431749 8129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:14.432063 8129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:14.432459 8129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:14.432502 8129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.432528 8129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:14.432583 8129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.441231 8129 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:45071
I20260807 00:40:14.441291 8321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:45071 every 8 connection(s)
I20260807 00:40:14.441624 8129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:14.445648 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 8129
I20260807 00:40:14.445755 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:14.445956 1490 external_mini_cluster.cc:1428] Setting key 03e7c251b2ac25c0b901e4cce5db723e
Aug 07 00:40:14 dist-test-slave-h6n0 krb5kdc[7589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063214, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:14.452462 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.442614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:35823 (local address 127.1.116.190:35073)
0807 00:40:14.442717 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.442722 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.442983 (+ 261us) server_negotiation.cc:407] Connection header received
0807 00:40:14.443580 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.443581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.443596 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.443651 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:14.444606 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.445131 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.446532 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.446922 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.447024 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.449375 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:14.449377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:14.449377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:14.449387 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:14.451114 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.451518 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.451519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.451520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.451593 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.451843 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.451844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.451845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.451982 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:14.452048 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.452286 (+ 238us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.452379 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:40:14.452618 8328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.442598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35073 (local address 127.1.116.131:35823)
0807 00:40:14.442845 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.442883 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.443434 (+ 551us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.443725 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.443728 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.443998 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:14.444496 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.444500 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.445230 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.445231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.446405 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.446408 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.447040 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.447041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.447141 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.447790 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:14.447797 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:14.449270 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:14.451203 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:14.451208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:14.451210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:14.451411 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:14.451674 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:14.451674 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:14.451675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:14.451767 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:14.452052 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:14.452056 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:14.452158 (+ 102us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.452242 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.452404 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":139,"threads_started":1}
I20260807 00:40:14.453323 8324 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35073
I20260807 00:40:14.453408 8324 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:14.453595 8324 heartbeater.cc:511] Master 127.1.116.190:35073 requested a full tablet report, sending...
I20260807 00:40:14.454066 7656 ts_manager.cc:194] Registered new tserver with Master: 17d023f57271463b9d89aff3b4d37075 (127.1.116.131:45071)
I20260807 00:40:14.454432 7656 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:35823
I20260807 00:40:14.458406 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:40:14.465961 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.459743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34446 (local address 127.1.116.190:35073)
0807 00:40:14.459797 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.459798 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.459855 (+ 57us) server_negotiation.cc:407] Connection header received
0807 00:40:14.459911 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.459912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.459921 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.459972 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:14.460514 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.461054 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.461754 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.461952 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.462015 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.463264 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:14.463265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:14.463266 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:14.463272 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:14.464820 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.465165 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.465166 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.465166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.465228 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:14.465455 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:14.465456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:14.465456 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:14.465621 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:14.465715 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.465824 (+ 109us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.465889 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 00:40:14.467804 7656 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:14.469780 7656 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:14.477680 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.473310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53756 (local address 127.1.116.131:45071)
0807 00:40:14.473477 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.473482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.473494 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:14.473534 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.473535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.473551 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.473630 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:14.473805 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.475050 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.476580 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.477188 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.477300 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.477358 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.477560 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.477601 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260807 00:40:14.477689 8347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.472762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:45071 (local address 127.0.0.1:53756)
0807 00:40:14.473002 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.473045 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.473117 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.473631 (+ 514us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.473634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.473635 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:14.473716 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.473719 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.475176 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.475177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.476470 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.476473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.477310 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.477311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.477424 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.477436 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.477546 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.477612 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":142,"threads_started":1}
I20260807 00:40:14.478056 8344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.472762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37365 (local address 127.0.0.1:35790)
0807 00:40:14.473220 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.473231 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.473271 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.473665 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.473666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.473670 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:14.473749 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.473751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.475624 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.475626 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.476676 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.476678 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.477805 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.477806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.477873 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.477878 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.477928 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.477975 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260807 00:40:14.478406 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.473069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35790 (local address 127.1.116.129:37365)
0807 00:40:14.473366 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.473372 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.473386 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:40:14.473436 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.473439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.473459 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.473836 (+ 377us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:14.473942 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.475073 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.476768 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.477639 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.477705 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.477768 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.478258 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.478302 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":144,"threads_started":1}
I20260807 00:40:14.478832 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.473066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33612 (local address 127.1.116.130:40927)
0807 00:40:14.475254 (+ 2188us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.475263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.475280 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:40:14.475316 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.475317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.475331 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.475425 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:14.475547 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.476312 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.477650 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.478385 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.478467 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.478551 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.478694 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.478743 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2148,"thread_start_us":176,"threads_started":1}
I20260807 00:40:14.478725 8226 tablet_service.cc:1467] Processing CreateTablet for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a (DEFAULT_TABLE table=test-table [id=94c9ddd0f42e4d8caeecbd3b6a3caab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:14.479138 8226 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:14.479027 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.472781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:40927 (local address 127.0.0.1:33612)
0807 00:40:14.473477 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.473488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.473538 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.475427 (+ 1889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.475429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.475432 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:14.475486 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.475488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.476425 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.476426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.477540 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.477544 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.478490 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.478491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.478586 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.478592 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.478664 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.478721 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":675,"mutex_wait_us":1,"thread_start_us":136,"threads_started":1}
I20260807 00:40:14.479228 7829 tablet_service.cc:1467] Processing CreateTablet for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a (DEFAULT_TABLE table=test-table [id=94c9ddd0f42e4d8caeecbd3b6a3caab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:14.479655 7829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:14.479938 8027 tablet_service.cc:1467] Processing CreateTablet for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a (DEFAULT_TABLE table=test-table [id=94c9ddd0f42e4d8caeecbd3b6a3caab4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:14.480278 8027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:14.482492 8356 tablet_bootstrap.cc:492] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3: Bootstrap starting.
I20260807 00:40:14.483197 8356 tablet_bootstrap.cc:654] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:14.483399 8358 tablet_bootstrap.cc:492] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Bootstrap starting.
I20260807 00:40:14.483780 8360 tablet_bootstrap.cc:492] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Bootstrap starting.
I20260807 00:40:14.484190 8358 tablet_bootstrap.cc:654] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:14.484202 8356 log.cc:824] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:14.484529 8360 tablet_bootstrap.cc:654] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:14.484871 8358 log.cc:824] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:14.485147 8360 log.cc:824] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:14.485327 8356 tablet_bootstrap.cc:492] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3: No bootstrap required, opened a new log
I20260807 00:40:14.485425 8356 ts_tablet_manager.cc:1397] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 00:40:14.485757 8358 tablet_bootstrap.cc:492] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: No bootstrap required, opened a new log
I20260807 00:40:14.485842 8358 ts_tablet_manager.cc:1397] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:40:14.486032 8360 tablet_bootstrap.cc:492] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: No bootstrap required, opened a new log
I20260807 00:40:14.486109 8360 ts_tablet_manager.cc:1397] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:40:14.487362 8356 raft_consensus.cc:348] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.487686 8356 raft_consensus.cc:374] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:14.487739 8356 raft_consensus.cc:729] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50d03cd73d29410b9bea999252f5c8e3, State: Initialized, Role: FOLLOWER
I20260807 00:40:14.487879 8356 consensus_queue.cc:260] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.487866 8358 raft_consensus.cc:348] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.488060 8358 raft_consensus.cc:374] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:14.488128 8358 raft_consensus.cc:729] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17d023f57271463b9d89aff3b4d37075, State: Initialized, Role: FOLLOWER
I20260807 00:40:14.488246 8356 ts_tablet_manager.cc:1428] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 00:40:14.488160 8360 raft_consensus.cc:348] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.488301 8360 raft_consensus.cc:374] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:14.488261 8358 consensus_queue.cc:260] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.488345 8360 raft_consensus.cc:729] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c88b499aeabd4d309f338c1876ad7620, State: Initialized, Role: FOLLOWER
I20260807 00:40:14.488389 7921 heartbeater.cc:503] Master 127.1.116.190:35073 was elected leader, sending a full tablet report...
I20260807 00:40:14.488480 8360 consensus_queue.cc:260] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.488547 8358 ts_tablet_manager.cc:1428] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:40:14.488662 8324 heartbeater.cc:503] Master 127.1.116.190:35073 was elected leader, sending a full tablet report...
I20260807 00:40:14.488775 8360 ts_tablet_manager.cc:1428] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:40:14.489192 8123 heartbeater.cc:503] Master 127.1.116.190:35073 was elected leader, sending a full tablet report...
W20260807 00:40:14.510704 8124 tablet.cc:2369] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:14.573567 8366 raft_consensus.cc:482] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:14.573741 8366 raft_consensus.cc:504] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.574306 8366 leader_election.cc:288] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17d023f57271463b9d89aff3b4d37075 (127.1.116.131:45071), c88b499aeabd4d309f338c1876ad7620 (127.1.116.130:40927)
I20260807 00:40:14.577410 8354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.574659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:49409 (local address 127.1.116.130:40927)
0807 00:40:14.574714 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.574717 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.574730 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:14.574767 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.574767 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.574778 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.574830 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:14.575014 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.575552 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.576553 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.577035 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.577124 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.577135 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.577292 (+ 157us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.577334 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 00:40:14.577410 7923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.574535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:40927 (local address 127.1.116.129:49409)
0807 00:40:14.574630 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.574645 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.574704 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.574832 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.574834 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.574835 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:14.574931 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.574933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.575646 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.575647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.576453 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.576455 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.577122 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.577123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.577213 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.577216 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.577285 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.577334 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66,"mutex_wait_us":4}
I20260807 00:40:14.577714 8355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.574689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:35119 (local address 127.1.116.131:45071)
0807 00:40:14.574741 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.574743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.574892 (+ 149us) server_negotiation.cc:407] Connection header received
0807 00:40:14.575001 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.575002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.575010 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.575080 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:14.575292 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.575773 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.576776 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.577210 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.577314 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.577323 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.577584 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.577639 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:40:14.577843 8055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da3fc4bd1814cb3b24a5d0c2369ea4a" candidate_uuid: "50d03cd73d29410b9bea999252f5c8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88b499aeabd4d309f338c1876ad7620" is_pre_election: true
I20260807 00:40:14.578020 8372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.574625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:45071 (local address 127.1.116.129:35119)
0807 00:40:14.574797 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.574813 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.574908 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.575088 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.575089 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.575111 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:14.575207 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.575209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.575876 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.575877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.576671 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:14.576673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.577331 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.577333 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:14.577447 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.577452 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.577843 (+ 391us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.577899 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260807 00:40:14.578035 8055 raft_consensus.cc:2454] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d03cd73d29410b9bea999252f5c8e3 in term 0.
I20260807 00:40:14.578399 7765 leader_election.cc:302] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d03cd73d29410b9bea999252f5c8e3, c88b499aeabd4d309f338c1876ad7620; no voters:
I20260807 00:40:14.578446 8259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da3fc4bd1814cb3b24a5d0c2369ea4a" candidate_uuid: "50d03cd73d29410b9bea999252f5c8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17d023f57271463b9d89aff3b4d37075" is_pre_election: true
I20260807 00:40:14.578672 8259 raft_consensus.cc:2454] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 50d03cd73d29410b9bea999252f5c8e3 in term 0.
I20260807 00:40:14.578790 8366 raft_consensus.cc:2781] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:14.578895 8366 raft_consensus.cc:482] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:14.578928 8366 raft_consensus.cc:3037] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:14.579841 8366 raft_consensus.cc:504] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.580021 8366 leader_election.cc:288] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [CANDIDATE]: Term 1 election: Requested vote from peers 17d023f57271463b9d89aff3b4d37075 (127.1.116.131:45071), c88b499aeabd4d309f338c1876ad7620 (127.1.116.130:40927)
I20260807 00:40:14.580250 8259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da3fc4bd1814cb3b24a5d0c2369ea4a" candidate_uuid: "50d03cd73d29410b9bea999252f5c8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17d023f57271463b9d89aff3b4d37075"
I20260807 00:40:14.580401 8259 raft_consensus.cc:3037] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:14.580430 8055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1da3fc4bd1814cb3b24a5d0c2369ea4a" candidate_uuid: "50d03cd73d29410b9bea999252f5c8e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c88b499aeabd4d309f338c1876ad7620"
I20260807 00:40:14.580500 8055 raft_consensus.cc:3037] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:14.581346 8259 raft_consensus.cc:2454] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d03cd73d29410b9bea999252f5c8e3 in term 1.
I20260807 00:40:14.581419 8055 raft_consensus.cc:2454] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 50d03cd73d29410b9bea999252f5c8e3 in term 1.
I20260807 00:40:14.581574 7765 leader_election.cc:302] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 50d03cd73d29410b9bea999252f5c8e3, c88b499aeabd4d309f338c1876ad7620; no voters:
I20260807 00:40:14.581689 8366 raft_consensus.cc:2781] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:14.581884 8366 raft_consensus.cc:686] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [term 1 LEADER]: Becoming Leader. State: Replica: 50d03cd73d29410b9bea999252f5c8e3, State: Running, Role: LEADER
I20260807 00:40:14.581998 8366 consensus_queue.cc:237] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } }
I20260807 00:40:14.583005 7656 catalog_manager.cc:5495] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 50d03cd73d29410b9bea999252f5c8e3 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "50d03cd73d29410b9bea999252f5c8e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "50d03cd73d29410b9bea999252f5c8e3" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 } health_report { overall_health: UNKNOWN } } }
W20260807 00:40:14.587986 7922 tablet.cc:2369] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:14.596115 8350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.593488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.1.116.129:37365)
0807 00:40:14.593542 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.593544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.593563 (+ 19us) server_negotiation.cc:407] Connection header received
0807 00:40:14.593596 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.593596 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.593614 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.593700 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:14.593832 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.594494 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.595322 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:14.595520 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:14.595606 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:14.595780 (+ 174us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:14.595876 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:14.595962 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.596004 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.596024 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 00:40:14.598284 8259 raft_consensus.cc:1261] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 1 FOLLOWER]: Refusing update from remote peer 50d03cd73d29410b9bea999252f5c8e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:14.598284 8055 raft_consensus.cc:1261] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 1 FOLLOWER]: Refusing update from remote peer 50d03cd73d29410b9bea999252f5c8e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:14.598773 8373 consensus_queue.cc:1035] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c88b499aeabd4d309f338c1876ad7620" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:14.598940 8366 consensus_queue.cc:1035] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 50d03cd73d29410b9bea999252f5c8e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17d023f57271463b9d89aff3b4d37075" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 45071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:14.601405 8378 mvcc.cc:205] Tried to move back new op lower bound from 7315714926990934016 to 7315714926928809984. Current Snapshot: MvccSnapshot[applied={T|T < 7315714926990934016}]
I20260807 00:40:14.601608 8379 mvcc.cc:205] Tried to move back new op lower bound from 7315714926990934016 to 7315714926928809984. Current Snapshot: MvccSnapshot[applied={T|T < 7315714926990934016}]
I20260807 00:40:14.604921 7656 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:14.605003 7656 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:14.605759 7656 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6e67c5aba6314fd7bd306b5b9a057b0b: Sending DeleteTablet for 3 replicas of tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a
I20260807 00:40:14.606191 7829 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:14 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:35790
I20260807 00:40:14.606379 8226 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:14 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:53756
I20260807 00:40:14.606748 8396 tablet_replica.cc:330] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: stopping tablet replica
I20260807 00:40:14.606742 8027 tablet_service.cc:1514] Processing DeleteTablet for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:14 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:33612
I20260807 00:40:14.606877 8396 raft_consensus.cc:2229] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:14.607110 8396 raft_consensus.cc:2258] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:14.607259 8397 tablet_replica.cc:330] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: stopping tablet replica
I20260807 00:40:14.607347 8397 raft_consensus.cc:2229] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:14.607462 8397 raft_consensus.cc:2258] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:14.607504 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 7742
I20260807 00:40:14.620798 8396 ts_tablet_manager.cc:1905] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:14.621433 8397 ts_tablet_manager.cc:1905] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:14.623027 8396 ts_tablet_manager.cc:1918] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:14.623119 8396 log.cc:1196] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/1da3fc4bd1814cb3b24a5d0c2369ea4a
I20260807 00:40:14.623471 8396 ts_tablet_manager.cc:1939] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P 17d023f57271463b9d89aff3b4d37075: Deleting consensus metadata
I20260807 00:40:14.623570 8397 ts_tablet_manager.cc:1918] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:14.623652 8397 log.cc:1196] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/1da3fc4bd1814cb3b24a5d0c2369ea4a
I20260807 00:40:14.623975 8397 ts_tablet_manager.cc:1939] T 1da3fc4bd1814cb3b24a5d0c2369ea4a P c88b499aeabd4d309f338c1876ad7620: Deleting consensus metadata
I20260807 00:40:14.624106 7638 catalog_manager.cc:4841] TS 17d023f57271463b9d89aff3b4d37075 (127.1.116.131:45071): tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a (table test-table [id=94c9ddd0f42e4d8caeecbd3b6a3caab4]) successfully deleted
I20260807 00:40:14.624481 7634 catalog_manager.cc:4841] TS c88b499aeabd4d309f338c1876ad7620 (127.1.116.130:40927): tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a (table test-table [id=94c9ddd0f42e4d8caeecbd3b6a3caab4]) successfully deleted
W20260807 00:40:14.625659 7639 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:37365 (error 108)
I20260807 00:40:14.626368 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.626193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37365 (local address 127.0.0.1:35804)
0807 00:40:14.626272 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.626332 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 00:40:14.626557 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 7928
W20260807 00:40:14.632323 7639 catalog_manager.cc:4584] TS 50d03cd73d29410b9bea999252f5c8e3 (127.1.116.129:37365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1da3fc4bd1814cb3b24a5d0c2369ea4a: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37365: connect: Connection refused (error 111)
I20260807 00:40:14.636169 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 8129
I20260807 00:40:14.645728 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 7615
2026-08-07T00:40:14Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2299 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T00:40:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:14Z Disabled control of system clock
I20260807 00:40:14.686604 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:44769
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:44769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 00:40:14.771649 8406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:14.771842 8406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:14.771862 8406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:14.773002 8406 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:14.773047 8406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:14.773061 8406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:14.773073 8406 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:14.773087 8406 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:14.774391 8406 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:44769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:44769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:14.774685 8406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:14.774995 8406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:14.780277 8411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.780328 8412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.780328 8415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:14.780748 8406 server_base.cc:1034] running on GCE node
I20260807 00:40:14.781003 8406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:14.781296 8406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:14.782733 8406 hybrid_clock.cc:648] HybridClock initialized: now 1786063214782501 us; error 39 us; skew 500 ppm
I20260807 00:40:14.784708 8406 webserver.cc:460] Webserver started at http://127.1.116.190:37109/ using document root <none> and password file <none>
I20260807 00:40:14.785034 8406 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:14.785136 8406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:14.785307 8406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:14.786634 8406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "31419764744147a88e4022ffa994ac64"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "3ff7c7714374713605a84ca2de364fcc"
server_key_iv: "15317d2e43e08825740cdf4cd6c33961"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.787070 8406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "31419764744147a88e4022ffa994ac64"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "3ff7c7714374713605a84ca2de364fcc"
server_key_iv: "15317d2e43e08825740cdf4cd6c33961"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.788836 8406 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260807 00:40:14.790009 8424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.790266 8406 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:14.790405 8406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "31419764744147a88e4022ffa994ac64"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "3ff7c7714374713605a84ca2de364fcc"
server_key_iv: "15317d2e43e08825740cdf4cd6c33961"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.790512 8406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:14.794104 8406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:14.794476 8406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:14.794642 8406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:14.799340 8406 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:44769
I20260807 00:40:14.799413 8500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:44769 every 8 connection(s)
I20260807 00:40:14.799767 8406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:14.800623 8501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:14.801267 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 8406
I20260807 00:40:14.801414 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:14.801616 1490 external_mini_cluster.cc:1428] Setting key 15dded5b695e5b1c2f826688f41c65e6
I20260807 00:40:14.804824 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.802329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47762 (local address 127.1.116.190:44769)
0807 00:40:14.802646 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.802657 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.802700 (+ 43us) server_negotiation.cc:407] Connection header received
0807 00:40:14.803283 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.803294 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.803358 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.803547 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:14.804306 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:14.804310 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:14.804311 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:14.804316 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:14.804371 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:14.804386 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:14.804473 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.804539 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.804608 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":99,"threads_started":1}
I20260807 00:40:14.805480 8501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64: Bootstrap starting.
I20260807 00:40:14.806178 8501 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:14.806522 8501 log.cc:824] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:14.807404 8501 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64: No bootstrap required, opened a new log
I20260807 00:40:14.809020 8501 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31419764744147a88e4022ffa994ac64" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44769 } }
I20260807 00:40:14.809196 8501 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:14.809223 8501 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31419764744147a88e4022ffa994ac64, State: Initialized, Role: FOLLOWER
I20260807 00:40:14.809401 8501 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31419764744147a88e4022ffa994ac64" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44769 } }
I20260807 00:40:14.809486 8501 raft_consensus.cc:388] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:14.809509 8501 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:14.809588 8501 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:14.810161 8501 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31419764744147a88e4022ffa994ac64" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44769 } }
I20260807 00:40:14.810285 8501 leader_election.cc:302] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 31419764744147a88e4022ffa994ac64; no voters:
I20260807 00:40:14.810508 8501 leader_election.cc:288] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:14.810880 8510 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:14.810914 8501 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:14.811105 8510 raft_consensus.cc:686] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [term 1 LEADER]: Becoming Leader. State: Replica: 31419764744147a88e4022ffa994ac64, State: Running, Role: LEADER
I20260807 00:40:14.811275 8510 consensus_queue.cc:237] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31419764744147a88e4022ffa994ac64" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44769 } }
I20260807 00:40:14.811792 8515 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 31419764744147a88e4022ffa994ac64. Latest consensus state: current_term: 1 leader_uuid: "31419764744147a88e4022ffa994ac64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31419764744147a88e4022ffa994ac64" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44769 } } }
I20260807 00:40:14.811945 8515 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:14.811988 8512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "31419764744147a88e4022ffa994ac64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "31419764744147a88e4022ffa994ac64" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44769 } } }
I20260807 00:40:14.812083 8512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64 [sys.catalog]: This master's current role is: LEADER
W20260807 00:40:14.813257 8535 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:40:14.813309 8535 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:14.813410 8523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:14.813529 8523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:14.815196 8523 catalog_manager.cc:1347] Generated new cluster ID: 438760e2daea4afaa5dd847e8fda5417
I20260807 00:40:14.815264 8523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:14.828855 8523 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:14.829008 8523 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:14.840363 8523 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64: Generated new TSK 0
I20260807 00:40:14.840538 8523 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:40:14.847311 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:44769
--builtin_ntp_servers=127.1.116.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 00:40:14.942591 8542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:14.942855 8542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:14.942893 8542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:14.944185 8542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:14.944247 8542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:14.944293 8542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:14.945714 8542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:14.946002 8542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:14.946414 8542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:14.952500 8550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.952500 8554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:14.952517 8552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:14.953306 8542 server_base.cc:1034] running on GCE node
I20260807 00:40:14.953588 8542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:14.953874 8542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:14.955054 8542 hybrid_clock.cc:648] HybridClock initialized: now 1786063214955039 us; error 43 us; skew 500 ppm
I20260807 00:40:14.956549 8542 webserver.cc:460] Webserver started at http://127.1.116.129:36667/ using document root <none> and password file <none>
I20260807 00:40:14.956835 8542 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:14.956908 8542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:14.957077 8542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:14.958300 8542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "220d90c35fdc49dd84641808331576fb"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "76ce37026a627e02e70a525546bfd1a4"
server_key_iv: "f260fe5a6bf4fe510b816ecaea8e998c"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.958738 8542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "220d90c35fdc49dd84641808331576fb"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "76ce37026a627e02e70a525546bfd1a4"
server_key_iv: "f260fe5a6bf4fe510b816ecaea8e998c"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.960228 8542 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:14.961108 8563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.961421 8542 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:14.961551 8542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "220d90c35fdc49dd84641808331576fb"
format_stamp: "Formatted at 2026-08-07 00:40:14 on dist-test-slave-h6n0"
server_key: "76ce37026a627e02e70a525546bfd1a4"
server_key_iv: "f260fe5a6bf4fe510b816ecaea8e998c"
server_key_version: "encryptionkey@0"
I20260807 00:40:14.961659 8542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:14.965579 8542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:14.965883 8542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:14.966028 8542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:14.966272 8542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:14.966681 8542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:14.966734 8542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.966775 8542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:14.966805 8542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:14.974381 8542 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:34685
I20260807 00:40:14.974434 8723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:34685 every 8 connection(s)
I20260807 00:40:14.974862 8542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:14.978817 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.975897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:33855 (local address 127.1.116.190:44769)
0807 00:40:14.975975 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:14.975978 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:14.976334 (+ 356us) server_negotiation.cc:407] Connection header received
0807 00:40:14.976939 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:14.976940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:14.976950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:14.977035 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:14.978484 (+ 1449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:14.978485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:14.978485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:14.978489 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:14.978503 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:14.978507 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:14.978647 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:14.978745 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 00:40:14.978759 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:40:14.979126 8727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:14.975896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:44769 (local address 127.1.116.129:33855)
0807 00:40:14.976106 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:14.976151 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:14.976819 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:14.977170 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:14.977174 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:14.977184 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:14.978308 (+ 1124us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:14.978309 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:14.978343 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:14.978381 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:14.978642 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:14.978647 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:14.978650 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:14.978733 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 00:40:14.978792 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260807 00:40:14.979926 8725 heartbeater.cc:347] Connected to a master server at 127.1.116.190:44769
I20260807 00:40:14.980247 8725 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:14.980494 8725 heartbeater.cc:511] Master 127.1.116.190:44769 requested a full tablet report, sending...
I20260807 00:40:14.981149 8449 ts_manager.cc:194] Registered new tserver with Master: 220d90c35fdc49dd84641808331576fb (127.1.116.129:34685)
I20260807 00:40:14.984875 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 8542
I20260807 00:40:14.984997 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:14.985224 1490 external_mini_cluster.cc:1428] Setting key 5ce41d2840485428cd20787f6c95fb8e
I20260807 00:40:14.986282 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:44769
--builtin_ntp_servers=127.1.116.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 00:40:15.074991 8730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:15.075197 8730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:15.075222 8730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:15.076324 8730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:15.076367 8730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:15.076395 8730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:15.077637 8730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:15.077847 8730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:15.078114 8730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:15.083392 8740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:15.083477 8738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:15.083403 8735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:15.083700 8730 server_base.cc:1034] running on GCE node
I20260807 00:40:15.083966 8730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:15.084223 8730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:15.085376 8730 hybrid_clock.cc:648] HybridClock initialized: now 1786063215085345 us; error 49 us; skew 500 ppm
I20260807 00:40:15.086874 8730 webserver.cc:460] Webserver started at http://127.1.116.130:40069/ using document root <none> and password file <none>
I20260807 00:40:15.087174 8730 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:15.087250 8730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:15.087422 8730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:15.088660 8730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "245b233624ff486daa88ac4118df0ed6"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "7f3340645f97a356d7e413cd7ef413e2"
server_key_iv: "f106c842f6303938d47e159e899bd79b"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.089063 8730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "245b233624ff486daa88ac4118df0ed6"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "7f3340645f97a356d7e413cd7ef413e2"
server_key_iv: "f106c842f6303938d47e159e899bd79b"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.090680 8730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:15.091550 8750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.091766 8730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:15.091882 8730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "245b233624ff486daa88ac4118df0ed6"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "7f3340645f97a356d7e413cd7ef413e2"
server_key_iv: "f106c842f6303938d47e159e899bd79b"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.091969 8730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:15.095793 8730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:15.096091 8730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:15.096246 8730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:15.096481 8730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:15.096819 8730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:15.096875 8730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.096915 8730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:15.096943 8730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.103974 8730 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:40265
I20260807 00:40:15.104050 8903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:40265 every 8 connection(s)
I20260807 00:40:15.104416 8730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:15.107650 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.105234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:42931 (local address 127.1.116.190:44769)
0807 00:40:15.105314 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.105316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.105557 (+ 241us) server_negotiation.cc:407] Connection header received
0807 00:40:15.106032 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.106033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.106041 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.106102 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.107389 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.107391 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.107391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.107396 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.107412 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.107417 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.107478 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.107567 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.107582 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:40:15.107786 8908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.105217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:44769 (local address 127.1.116.130:42931)
0807 00:40:15.105447 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.105478 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.105926 (+ 448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.106181 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.106184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.106192 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:15.107227 (+ 1035us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:15.107228 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:15.107254 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:15.107281 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:15.107493 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:15.107496 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:15.107498 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:40:15.107557 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:40:15.107611 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":103,"threads_started":1}
I20260807 00:40:15.108392 8906 heartbeater.cc:347] Connected to a master server at 127.1.116.190:44769
I20260807 00:40:15.108486 8906 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:15.108686 8906 heartbeater.cc:511] Master 127.1.116.190:44769 requested a full tablet report, sending...
I20260807 00:40:15.109154 8449 ts_manager.cc:194] Registered new tserver with Master: 245b233624ff486daa88ac4118df0ed6 (127.1.116.130:40265)
I20260807 00:40:15.112756 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 8730
I20260807 00:40:15.112843 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:15.113052 1490 external_mini_cluster.cc:1428] Setting key 55196a4e75bd897cfdce39e754de39c8
I20260807 00:40:15.114050 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:44769
--builtin_ntp_servers=127.1.116.148:34999
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 00:40:15.206012 8909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:15.206247 8909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:15.206271 8909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:15.207413 8909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:15.207454 8909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:15.207480 8909 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:15.208849 8909 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34999
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:44769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:15.209076 8909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:15.209357 8909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:15.215119 8922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:15.215119 8920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:15.215248 8924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:15.215596 8909 server_base.cc:1034] running on GCE node
I20260807 00:40:15.215829 8909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:15.216096 8909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:15.217293 8909 hybrid_clock.cc:648] HybridClock initialized: now 1786063215217276 us; error 38 us; skew 500 ppm
I20260807 00:40:15.218856 8909 webserver.cc:460] Webserver started at http://127.1.116.131:37911/ using document root <none> and password file <none>
I20260807 00:40:15.219182 8909 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:15.219264 8909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:15.219431 8909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:15.220604 8909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "dc143db9498f48a2b7f6bc7bd85fdd36"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "a7632cee949dd241f241aba131f65a1c"
server_key_iv: "7f6df42ab6cd9cfcfdb325bf198a80b3"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.221001 8909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "dc143db9498f48a2b7f6bc7bd85fdd36"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "a7632cee949dd241f241aba131f65a1c"
server_key_iv: "7f6df42ab6cd9cfcfdb325bf198a80b3"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.222473 8909 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:15.223418 8933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.223664 8909 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:15.223788 8909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "dc143db9498f48a2b7f6bc7bd85fdd36"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "a7632cee949dd241f241aba131f65a1c"
server_key_iv: "7f6df42ab6cd9cfcfdb325bf198a80b3"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.223867 8909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:15.227730 8909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:15.228021 8909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:15.228184 8909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:15.228411 8909 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:15.228754 8909 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:15.228814 8909 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.228856 8909 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:15.228885 8909 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.236359 8909 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:46445
I20260807 00:40:15.236416 9105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:46445 every 8 connection(s)
I20260807 00:40:15.236783 8909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:15.240358 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.237666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:37197 (local address 127.1.116.190:44769)
0807 00:40:15.237744 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.237765 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.238026 (+ 261us) server_negotiation.cc:407] Connection header received
0807 00:40:15.238743 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.238744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.238751 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.238816 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.240080 (+ 1264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.240082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.240082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.240087 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.240104 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.240109 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.240181 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.240265 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.240281 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:40:15.240582 9108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.237621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:44769 (local address 127.1.116.131:37197)
0807 00:40:15.237872 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.237908 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.238630 (+ 722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.238889 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.238894 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.238903 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:15.239906 (+ 1003us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:15.239906 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:15.239936 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:15.239966 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:15.240194 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:15.240200 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:15.240203 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:15.240277 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 00:40:15.240340 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":111,"threads_started":1}
I20260807 00:40:15.240710 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 8909
I20260807 00:40:15.240836 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:15.241011 1490 external_mini_cluster.cc:1428] Setting key 8d4906c4beb7f86bd86b818b1bdc7036
I20260807 00:40:15.242394 9106 heartbeater.cc:347] Connected to a master server at 127.1.116.190:44769
I20260807 00:40:15.242488 9106 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:15.242699 9106 heartbeater.cc:511] Master 127.1.116.190:44769 requested a full tablet report, sending...
I20260807 00:40:15.243109 8449 ts_manager.cc:194] Registered new tserver with Master: dc143db9498f48a2b7f6bc7bd85fdd36 (127.1.116.131:46445)
I20260807 00:40:15.253549 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:40:15.256321 8506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.255061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47768 (local address 127.1.116.190:44769)
0807 00:40:15.255109 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.255111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.255290 (+ 179us) server_negotiation.cc:407] Connection header received
0807 00:40:15.255326 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.255327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.255335 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.255389 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.255999 (+ 610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.256000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.256001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.256005 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.256021 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.256025 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.256070 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.256184 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.256206 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:40:15.258328 8449 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47768:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:15.260277 8449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:15.265152 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.263270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:40265 (local address 127.0.0.1:33884)
0807 00:40:15.263879 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.263891 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.263937 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.264180 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.264181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.264184 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:15.264671 (+ 487us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:15.264672 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:15.264695 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:15.264718 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:15.264964 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:15.264971 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:15.264974 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:15.265048 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 00:40:15.265065 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":84,"threads_started":1}
I20260807 00:40:15.265261 9120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.263409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33884 (local address 127.1.116.130:40265)
0807 00:40:15.263657 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.263661 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.263999 (+ 338us) server_negotiation.cc:407] Connection header received
0807 00:40:15.264033 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.264035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.264047 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.264368 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.264807 (+ 439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.264814 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.264814 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.264818 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.264856 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.264877 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.264933 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.265097 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.265132 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":106,"threads_started":1}
I20260807 00:40:15.265583 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.263258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41838 (local address 127.1.116.129:34685)
0807 00:40:15.263502 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.263506 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.263539 (+ 33us) server_negotiation.cc:407] Connection header received
0807 00:40:15.263601 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.263604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.263616 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.263705 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.264915 (+ 1210us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.264924 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.264924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.264929 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.264970 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.264999 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.265413 (+ 414us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.265437 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.265462 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
I20260807 00:40:15.265599 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.263017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:46445 (local address 127.0.0.1:35578)
0807 00:40:15.263226 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.263579 (+ 353us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.263742 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.263981 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.263982 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.263983 (+ 1us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:15.265109 (+ 1126us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:15.265110 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:15.265118 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:15.265130 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:15.265446 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:15.265447 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:15.265448 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 00:40:15.265504 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 00:40:15.265520 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":84,"threads_started":1}
I20260807 00:40:15.265709 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.263158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35578 (local address 127.1.116.131:46445)
0807 00:40:15.263418 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.263425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.263742 (+ 317us) server_negotiation.cc:407] Connection header received
0807 00:40:15.263812 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.263816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.263835 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.263927 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.265224 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.265233 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.265233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.265237 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.265273 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.265297 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.265392 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.265547 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.265568 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":134,"threads_started":1}
I20260807 00:40:15.266393 8819 tablet_service.cc:1467] Processing CreateTablet for tablet d70aa1d565a14130af758335dce1a26a (DEFAULT_TABLE table=test-table [id=4045cfe8ad024fb8be7dd4ef7aab6cff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:15.266398 9012 tablet_service.cc:1467] Processing CreateTablet for tablet d70aa1d565a14130af758335dce1a26a (DEFAULT_TABLE table=test-table [id=4045cfe8ad024fb8be7dd4ef7aab6cff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:15.265336 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.263018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:34685 (local address 127.0.0.1:41838)
0807 00:40:15.263411 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.263454 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.263521 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.263844 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.263916 (+ 72us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.263925 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:15.264815 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:15.264815 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:15.264826 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:15.264846 (+ 20us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:15.265214 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:15.265215 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:15.265216 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 00:40:15.265243 (+ 27us) client_negotiation.cc:240] Negotiation successful
0807 00:40:15.265257 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260807 00:40:15.266824 8819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d70aa1d565a14130af758335dce1a26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:15.266824 9012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d70aa1d565a14130af758335dce1a26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:15.267503 8624 tablet_service.cc:1467] Processing CreateTablet for tablet d70aa1d565a14130af758335dce1a26a (DEFAULT_TABLE table=test-table [id=4045cfe8ad024fb8be7dd4ef7aab6cff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:15.267836 8624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d70aa1d565a14130af758335dce1a26a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:15.270328 9122 tablet_bootstrap.cc:492] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Bootstrap starting.
I20260807 00:40:15.271190 9126 tablet_bootstrap.cc:492] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb: Bootstrap starting.
I20260807 00:40:15.271456 9122 tablet_bootstrap.cc:654] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:15.271811 9122 log.cc:824] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:15.271883 9126 tablet_bootstrap.cc:654] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:15.272223 9124 tablet_bootstrap.cc:492] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Bootstrap starting.
I20260807 00:40:15.272235 9126 log.cc:824] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:15.272866 9124 tablet_bootstrap.cc:654] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:15.273013 9126 tablet_bootstrap.cc:492] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb: No bootstrap required, opened a new log
I20260807 00:40:15.273085 9126 ts_tablet_manager.cc:1397] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:15.273270 9124 log.cc:824] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:15.273841 9122 tablet_bootstrap.cc:492] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: No bootstrap required, opened a new log
I20260807 00:40:15.273939 9122 ts_tablet_manager.cc:1397] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 00:40:15.274179 9124 tablet_bootstrap.cc:492] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: No bootstrap required, opened a new log
I20260807 00:40:15.274252 9124 ts_tablet_manager.cc:1397] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:15.275691 9126 raft_consensus.cc:348] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.275854 9126 raft_consensus.cc:374] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:15.275897 9126 raft_consensus.cc:729] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220d90c35fdc49dd84641808331576fb, State: Initialized, Role: FOLLOWER
I20260807 00:40:15.276026 9126 consensus_queue.cc:260] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.276314 9126 ts_tablet_manager.cc:1428] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 00:40:15.276476 8725 heartbeater.cc:503] Master 127.1.116.190:44769 was elected leader, sending a full tablet report...
I20260807 00:40:15.277240 9124 raft_consensus.cc:348] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.277205 9122 raft_consensus.cc:348] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.277400 9122 raft_consensus.cc:374] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:15.277400 9124 raft_consensus.cc:374] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:15.277446 9124 raft_consensus.cc:729] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 245b233624ff486daa88ac4118df0ed6, State: Initialized, Role: FOLLOWER
I20260807 00:40:15.277446 9122 raft_consensus.cc:729] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc143db9498f48a2b7f6bc7bd85fdd36, State: Initialized, Role: FOLLOWER
I20260807 00:40:15.277572 9122 consensus_queue.cc:260] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.277572 9124 consensus_queue.cc:260] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.277871 9124 ts_tablet_manager.cc:1428] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 00:40:15.277880 9122 ts_tablet_manager.cc:1428] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 00:40:15.277918 8906 heartbeater.cc:503] Master 127.1.116.190:44769 was elected leader, sending a full tablet report...
I20260807 00:40:15.278019 9106 heartbeater.cc:503] Master 127.1.116.190:44769 was elected leader, sending a full tablet report...
W20260807 00:40:15.355063 8907 tablet.cc:2369] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:40:15.475733 8726 tablet.cc:2369] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:15.480777 9132 raft_consensus.cc:482] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:15.480906 9132 raft_consensus.cc:504] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.481505 9132 leader_election.cc:288] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc143db9498f48a2b7f6bc7bd85fdd36 (127.1.116.131:46445), 245b233624ff486daa88ac4118df0ed6 (127.1.116.130:40265)
I20260807 00:40:15.483523 9140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.481983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:40265 (local address 127.1.116.129:46753)
0807 00:40:15.482111 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.482124 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.482191 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.482376 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.482378 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.482381 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:15.483012 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:15.483012 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:15.483023 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:15.483042 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:15.483331 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:15.483332 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:15.483333 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 00:40:15.483401 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:40:15.483421 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":44,"threads_started":1}
I20260807 00:40:15.483567 9120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.482077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:46753 (local address 127.1.116.130:40265)
0807 00:40:15.482127 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.482129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.482188 (+ 59us) server_negotiation.cc:407] Connection header received
0807 00:40:15.482279 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.482280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.482291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.482355 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.483166 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.483168 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.483168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.483176 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.483206 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.483214 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.483308 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.483419 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.483436 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 00:40:15.483686 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.481831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:49211 (local address 127.1.116.131:46445)
0807 00:40:15.481924 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.481927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.482105 (+ 178us) server_negotiation.cc:407] Connection header received
0807 00:40:15.482282 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.482284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.482293 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.482332 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.483365 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.483367 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.483367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.483372 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.483395 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.483401 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.483466 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.483557 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.483573 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:40:15.483920 9135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.481759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:46445 (local address 127.1.116.129:49211)
0807 00:40:15.481975 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.481993 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.482149 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.482443 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.482444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.482478 (+ 34us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:15.483234 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:15.483235 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:15.483246 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:15.483263 (+ 17us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:15.483485 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:15.483486 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:15.483487 (+ 1us) client_negotiation.cc:769] Sending connection context
0807 00:40:15.483564 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 00:40:15.483806 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":103,"threads_started":1}
I20260807 00:40:15.483945 8847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70aa1d565a14130af758335dce1a26a" candidate_uuid: "220d90c35fdc49dd84641808331576fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245b233624ff486daa88ac4118df0ed6" is_pre_election: true
I20260807 00:40:15.484140 8847 raft_consensus.cc:2454] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220d90c35fdc49dd84641808331576fb in term 0.
I20260807 00:40:15.484364 9040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70aa1d565a14130af758335dce1a26a" candidate_uuid: "220d90c35fdc49dd84641808331576fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" is_pre_election: true
I20260807 00:40:15.484457 8564 leader_election.cc:302] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220d90c35fdc49dd84641808331576fb, 245b233624ff486daa88ac4118df0ed6; no voters:
I20260807 00:40:15.484508 9040 raft_consensus.cc:2454] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 220d90c35fdc49dd84641808331576fb in term 0.
I20260807 00:40:15.484642 9132 raft_consensus.cc:2781] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:15.484704 9132 raft_consensus.cc:482] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:15.484732 9132 raft_consensus.cc:3037] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:15.485627 9132 raft_consensus.cc:504] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.485775 9132 leader_election.cc:288] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [CANDIDATE]: Term 1 election: Requested vote from peers dc143db9498f48a2b7f6bc7bd85fdd36 (127.1.116.131:46445), 245b233624ff486daa88ac4118df0ed6 (127.1.116.130:40265)
I20260807 00:40:15.485988 9040 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70aa1d565a14130af758335dce1a26a" candidate_uuid: "220d90c35fdc49dd84641808331576fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36"
I20260807 00:40:15.485982 8847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d70aa1d565a14130af758335dce1a26a" candidate_uuid: "220d90c35fdc49dd84641808331576fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "245b233624ff486daa88ac4118df0ed6"
I20260807 00:40:15.486106 9040 raft_consensus.cc:3037] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:15.486097 8847 raft_consensus.cc:3037] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:15.487023 9040 raft_consensus.cc:2454] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220d90c35fdc49dd84641808331576fb in term 1.
I20260807 00:40:15.487226 8847 raft_consensus.cc:2454] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 220d90c35fdc49dd84641808331576fb in term 1.
I20260807 00:40:15.487298 8564 leader_election.cc:302] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220d90c35fdc49dd84641808331576fb, dc143db9498f48a2b7f6bc7bd85fdd36; no voters:
W20260807 00:40:15.487360 9107 tablet.cc:2369] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:15.487414 9132 raft_consensus.cc:2781] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:15.487597 9132 raft_consensus.cc:686] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [term 1 LEADER]: Becoming Leader. State: Replica: 220d90c35fdc49dd84641808331576fb, State: Running, Role: LEADER
I20260807 00:40:15.487708 9132 consensus_queue.cc:237] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } }
I20260807 00:40:15.488838 8448 catalog_manager.cc:5495] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb reported cstate change: term changed from 0 to 1, leader changed from <none> to 220d90c35fdc49dd84641808331576fb (127.1.116.129). New cstate: current_term: 1 leader_uuid: "220d90c35fdc49dd84641808331576fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220d90c35fdc49dd84641808331576fb" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 34685 } health_report { overall_health: HEALTHY } } }
I20260807 00:40:15.550195 9119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.548913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.1.116.129:34685)
0807 00:40:15.549025 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.549027 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.549043 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:40:15.549101 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.549102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.549126 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.549242 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.549887 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.549888 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.549888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.549892 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.549909 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.549913 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.549970 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.550094 (+ 124us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.550107 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 00:40:15.552500 8847 raft_consensus.cc:1261] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 1 FOLLOWER]: Refusing update from remote peer 220d90c35fdc49dd84641808331576fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:15.552551 9040 raft_consensus.cc:1261] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 1 FOLLOWER]: Refusing update from remote peer 220d90c35fdc49dd84641808331576fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:15.552958 9141 consensus_queue.cc:1035] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "245b233624ff486daa88ac4118df0ed6" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 40265 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:15.553117 9132 consensus_queue.cc:1035] T d70aa1d565a14130af758335dce1a26a P 220d90c35fdc49dd84641808331576fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc143db9498f48a2b7f6bc7bd85fdd36" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 46445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:15.557593 9146 mvcc.cc:205] Tried to move back new op lower bound from 7315714930899890176 to 7315714930638594048. Current Snapshot: MvccSnapshot[applied={T|T < 7315714930899890176 or (T in {7315714930899890176})}]
I20260807 00:40:15.558609 8448 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:15.558717 8448 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47768:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:15.559640 8448 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 31419764744147a88e4022ffa994ac64: Sending DeleteTablet for 3 replicas of tablet d70aa1d565a14130af758335dce1a26a
I20260807 00:40:15.560009 8624 tablet_service.cc:1514] Processing DeleteTablet for tablet d70aa1d565a14130af758335dce1a26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:15 UTC) from {username='slave'} at 127.0.0.1:41838
I20260807 00:40:15.560078 9012 tablet_service.cc:1514] Processing DeleteTablet for tablet d70aa1d565a14130af758335dce1a26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:15 UTC) from {username='slave'} at 127.0.0.1:35578
I20260807 00:40:15.560113 8819 tablet_service.cc:1514] Processing DeleteTablet for tablet d70aa1d565a14130af758335dce1a26a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:15 UTC) from {username='slave'} at 127.0.0.1:33884
I20260807 00:40:15.560305 9163 tablet_replica.cc:330] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: stopping tablet replica
I20260807 00:40:15.560407 9163 raft_consensus.cc:2229] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:15.560526 9163 raft_consensus.cc:2258] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:15.560741 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 8542
I20260807 00:40:15.567096 9169 tablet_replica.cc:330] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: stopping tablet replica
I20260807 00:40:15.567263 9169 raft_consensus.cc:2229] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:15.567407 9169 raft_consensus.cc:2258] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:15.567656 9163 ts_tablet_manager.cc:1905] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:15.567963 9169 ts_tablet_manager.cc:1905] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:15.569746 9169 ts_tablet_manager.cc:1918] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:15.569746 9163 ts_tablet_manager.cc:1918] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:15.569834 9169 log.cc:1196] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/d70aa1d565a14130af758335dce1a26a
I20260807 00:40:15.569957 9163 log.cc:1196] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/d70aa1d565a14130af758335dce1a26a
I20260807 00:40:15.570144 9169 ts_tablet_manager.cc:1939] T d70aa1d565a14130af758335dce1a26a P 245b233624ff486daa88ac4118df0ed6: Deleting consensus metadata
I20260807 00:40:15.570371 9163 ts_tablet_manager.cc:1939] T d70aa1d565a14130af758335dce1a26a P dc143db9498f48a2b7f6bc7bd85fdd36: Deleting consensus metadata
I20260807 00:40:15.570683 8426 catalog_manager.cc:4841] TS 245b233624ff486daa88ac4118df0ed6 (127.1.116.130:40265): tablet d70aa1d565a14130af758335dce1a26a (table test-table [id=4045cfe8ad024fb8be7dd4ef7aab6cff]) successfully deleted
I20260807 00:40:15.570847 8426 catalog_manager.cc:4841] TS dc143db9498f48a2b7f6bc7bd85fdd36 (127.1.116.131:46445): tablet d70aa1d565a14130af758335dce1a26a (table test-table [id=4045cfe8ad024fb8be7dd4ef7aab6cff]) successfully deleted
W20260807 00:40:15.573952 8428 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:34685 (error 108)
I20260807 00:40:15.574100 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 8730
I20260807 00:40:15.578842 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.574388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:34685 (local address 127.0.0.1:41856)
0807 00:40:15.578745 (+ 4357us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.578803 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:34685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4322}
W20260807 00:40:15.579007 8428 catalog_manager.cc:4584] TS 220d90c35fdc49dd84641808331576fb (127.1.116.129:34685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d70aa1d565a14130af758335dce1a26a: Network error: Client connection negotiation failed: client connection to 127.1.116.129:34685: connect: Connection refused (error 111)
I20260807 00:40:15.586153 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 8909
I20260807 00:40:15.595942 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 8406
2026-08-07T00:40:15Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (946 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 00:40:15.624042 1490 mini_oidc.cc:152] Starting JWKS server
I20260807 00:40:15.625360 1490 webserver.cc:460] Webserver started at https://127.0.0.1:32839/ using document root <none> and password file <none>
I20260807 00:40:15.625449 1490 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 00:40:15.625659 1490 webserver.cc:460] Webserver started at http://127.0.0.1:38193/ using document root <none> and password file <none>
2026-08-07T00:40:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:15Z Disabled control of system clock
I20260807 00:40:15.635655 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:44275
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:37649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:44275
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32839/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/testchainca.cert with env {}
W20260807 00:40:15.729772 9176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:15.730005 9176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:15.730027 9176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:15.731396 9176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:15.731444 9176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:15.731457 9176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:15.731470 9176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:15.731483 9176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:40:15.731534 9176 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:40:15.731554 9176 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:32839/jwks.json
I20260807 00:40:15.733153 9176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:37649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:44275
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:44275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:32839/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:15.733451 9176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:15.733767 9176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:15.739048 9182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:15.739064 9185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:15.739415 9176 server_base.cc:1034] running on GCE node
W20260807 00:40:15.739529 9181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:15.739979 9176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:15.740340 9176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:15.741530 9176 hybrid_clock.cc:648] HybridClock initialized: now 1786063215741501 us; error 54 us; skew 500 ppm
I20260807 00:40:15.743075 9176 webserver.cc:460] Webserver started at http://127.1.116.190:36105/ using document root <none> and password file <none>
I20260807 00:40:15.743379 9176 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:15.743454 9176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:15.743619 9176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:15.744843 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "0c00bcb8d49f4a41b06e3e876e2d9949"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "7dafd274c133f3e3b1256e3c3c2ba911"
server_key_iv: "7cfa1ad9df0db1be3cb92e2061ed06d1"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.745256 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "0c00bcb8d49f4a41b06e3e876e2d9949"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "7dafd274c133f3e3b1256e3c3c2ba911"
server_key_iv: "7cfa1ad9df0db1be3cb92e2061ed06d1"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.746825 9176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.003s
I20260807 00:40:15.747778 9198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.748106 9176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:15.748265 9176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "0c00bcb8d49f4a41b06e3e876e2d9949"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "7dafd274c133f3e3b1256e3c3c2ba911"
server_key_iv: "7cfa1ad9df0db1be3cb92e2061ed06d1"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.748349 9176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:15.761585 9176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:15.761950 9176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:15.762112 9176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:15.767390 9176 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:44275
I20260807 00:40:15.767448 9266 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:44275 every 8 connection(s)
I20260807 00:40:15.767861 9176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:15.769001 9267 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:15.771400 9267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949: Bootstrap starting.
I20260807 00:40:15.771741 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 9176
I20260807 00:40:15.771878 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:15.772086 1490 external_mini_cluster.cc:1428] Setting key 5785f85eeb19d9c99b0f44161601833b
I20260807 00:40:15.772190 9267 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:15.772571 9267 log.cc:824] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:15.773473 9267 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949: No bootstrap required, opened a new log
I20260807 00:40:15.775940 9267 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44275 } }
I20260807 00:40:15.776087 9267 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:15.776111 9267 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c00bcb8d49f4a41b06e3e876e2d9949, State: Initialized, Role: FOLLOWER
I20260807 00:40:15.776239 9267 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44275 } }
I20260807 00:40:15.776324 9267 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:15.776369 9267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:15.776422 9267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:15.777100 9267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44275 } }
I20260807 00:40:15.777213 9267 leader_election.cc:302] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c00bcb8d49f4a41b06e3e876e2d9949; no voters:
I20260807 00:40:15.777381 9267 leader_election.cc:288] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:15.777478 9272 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:15.777678 9267 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:15.777688 9272 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [term 1 LEADER]: Becoming Leader. State: Replica: 0c00bcb8d49f4a41b06e3e876e2d9949, State: Running, Role: LEADER
I20260807 00:40:15.777866 9272 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44275 } }
I20260807 00:40:15.778316 9272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c00bcb8d49f4a41b06e3e876e2d9949. Latest consensus state: current_term: 1 leader_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44275 } } }
I20260807 00:40:15.778411 9272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:15.778815 9273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c00bcb8d49f4a41b06e3e876e2d9949" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 44275 } } }
I20260807 00:40:15.778895 9273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:15.779224 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.772715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.1.116.190:44275)
0807 00:40:15.772961 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.772969 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.772993 (+ 24us) server_negotiation.cc:407] Connection header received
0807 00:40:15.773590 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.773601 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.773693 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.773840 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:15.774114 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:15.775598 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:15.776710 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:15.777342 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:15.777496 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:15.778389 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:15.778393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:15.778393 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:15.778616 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:15.778664 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:15.778679 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:15.778752 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:15.778891 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 00:40:15.778975 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":120,"threads_started":1}
I20260807 00:40:15.779624 9278 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:15.780181 9278 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:15.780717 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:40:15.782058 9278 catalog_manager.cc:1347] Generated new cluster ID: 936ef435b38944d48eac1866d13d9bbf
I20260807 00:40:15.782135 9278 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:15.789253 9278 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:15.789832 9278 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:15.793522 9278 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c00bcb8d49f4a41b06e3e876e2d9949: Generated new TSK 0
I20260807 00:40:15.793720 9278 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 00:40:15.829715 9311 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 00:40:15.829890 9311 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 00:40:15.830071 9271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:15.806566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34074 (local address 127.1.116.190:44275)
0807 00:40:15.806630 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:15.806632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:15.806782 (+ 150us) server_negotiation.cc:407] Connection header received
0807 00:40:15.806920 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:15.806921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:15.806941 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:15.807014 (+ 73us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:40:15.807180 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:15.807734 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:15.808590 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:15.808823 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:15.808905 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:15.809116 (+ 211us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:40:15.829751 (+ 20635us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 00:40:15.829981 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":33}
W20260807 00:40:15.830228 9271 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34074: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 00:40:15.830827 9310 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:15.806463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:44275 (local address 127.0.0.1:34074)
0807 00:40:15.806697 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:15.806709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:15.806809 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:15.807024 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:15.807025 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:15.807027 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:40:15.807095 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:15.807104 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:15.807829 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:15.807830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:15.808499 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:15.808501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:15.808907 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:15.808908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:15.808969 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:15.808997 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:40:15.830153 (+ 21156us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 00:40:15.830417 (+ 264us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.116.190:44275: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":130,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.116.190:44275: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:32839/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 00:40:15.831334 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 9176
2026-08-07T00:40:15Z chronyd exiting
I20260807 00:40:15.876451 1490 test_util.cc:175] -----------------------------------------------
I20260807 00:40:15.876531 1490 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786063190385254-1490-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (253 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 00:40:15.877413 1490 mini_oidc.cc:152] Starting JWKS server
I20260807 00:40:15.878067 1490 webserver.cc:460] Webserver started at https://127.0.0.1:40223/ using document root <none> and password file <none>
I20260807 00:40:15.878109 1490 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 00:40:15.878259 1490 webserver.cc:460] Webserver started at http://127.0.0.1:46729/ using document root <none> and password file <none>
2026-08-07T00:40:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:15Z Disabled control of system clock
I20260807 00:40:15.888907 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41767
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:45107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:41767
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40223/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/testchainca.cert with env {}
W20260807 00:40:15.973730 9320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:15.973958 9320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:15.974004 9320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:15.975772 9320 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:15.975837 9320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:15.975862 9320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:15.975893 9320 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:15.975920 9320 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:40:15.975945 9320 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:40:15.975973 9320 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40223/jwks.json
I20260807 00:40:15.977478 9320 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:41767
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40223/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:15.977733 9320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:15.977945 9320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:15.982999 9329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:15.982959 9328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:15.983188 9320 server_base.cc:1034] running on GCE node
W20260807 00:40:15.983006 9331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:15.983530 9320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:15.983939 9320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:15.985167 9320 hybrid_clock.cc:648] HybridClock initialized: now 1786063215985144 us; error 55 us; skew 500 ppm
I20260807 00:40:15.986759 9320 webserver.cc:460] Webserver started at http://127.1.116.190:34119/ using document root <none> and password file <none>
I20260807 00:40:15.987052 9320 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:15.987154 9320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:15.987314 9320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:15.988531 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "4730cbed0da643b091c9b4d18e1d1ad8"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "769487181a327fbe721b40c45c2d8721"
server_key_iv: "1387e29c85abf4e3169e4cd834c2bebf"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.988930 9320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "4730cbed0da643b091c9b4d18e1d1ad8"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "769487181a327fbe721b40c45c2d8721"
server_key_iv: "1387e29c85abf4e3169e4cd834c2bebf"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.990350 9320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:15.991292 9342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:15.991513 9320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:15.991631 9320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "4730cbed0da643b091c9b4d18e1d1ad8"
format_stamp: "Formatted at 2026-08-07 00:40:15 on dist-test-slave-h6n0"
server_key: "769487181a327fbe721b40c45c2d8721"
server_key_iv: "1387e29c85abf4e3169e4cd834c2bebf"
server_key_version: "encryptionkey@0"
I20260807 00:40:15.991714 9320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:16.003202 9320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:16.003547 9320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:16.003706 9320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:16.008607 9320 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:41767
I20260807 00:40:16.008652 9417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:41767 every 8 connection(s)
I20260807 00:40:16.009066 9320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:16.010144 9418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:16.012447 9418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8: Bootstrap starting.
I20260807 00:40:16.013032 9418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:16.013339 9418 log.cc:824] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:16.014217 9418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8: No bootstrap required, opened a new log
I20260807 00:40:16.014791 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 9320
I20260807 00:40:16.014925 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:16.015126 1490 external_mini_cluster.cc:1428] Setting key 5cbead323018559458316aee7607ad0b
I20260807 00:40:16.016297 9418 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41767 } }
I20260807 00:40:16.016471 9418 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:16.016495 9418 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4730cbed0da643b091c9b4d18e1d1ad8, State: Initialized, Role: FOLLOWER
I20260807 00:40:16.016688 9418 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41767 } }
I20260807 00:40:16.016777 9418 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:16.016809 9418 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:16.016868 9418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:16.017441 9418 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41767 } }
I20260807 00:40:16.017530 9418 leader_election.cc:302] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4730cbed0da643b091c9b4d18e1d1ad8; no voters:
I20260807 00:40:16.017741 9418 leader_election.cc:288] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:16.017774 9425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:16.017901 9425 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [term 1 LEADER]: Becoming Leader. State: Replica: 4730cbed0da643b091c9b4d18e1d1ad8, State: Running, Role: LEADER
I20260807 00:40:16.018011 9418 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:16.017990 9425 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41767 } }
I20260807 00:40:16.018327 9425 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4730cbed0da643b091c9b4d18e1d1ad8. Latest consensus state: current_term: 1 leader_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41767 } } }
I20260807 00:40:16.018424 9425 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:16.018337 9427 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4730cbed0da643b091c9b4d18e1d1ad8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41767 } } }
I20260807 00:40:16.018679 9427 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:16.018678 9432 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:16.019330 9432 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:16.021109 9432 catalog_manager.cc:1347] Generated new cluster ID: 19c9ed2c348a4b7b80c88f68e1f33e03
I20260807 00:40:16.021171 9432 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:16.022012 9424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:16.015725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.1.116.190:41767)
0807 00:40:16.015986 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:16.015994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:16.016023 (+ 29us) server_negotiation.cc:407] Connection header received
0807 00:40:16.016284 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:16.016292 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:16.016360 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:16.016520 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:16.016828 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.018511 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.019729 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.020355 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.020823 (+ 468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.021225 (+ 402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:16.021229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:16.021230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:16.021443 (+ 213us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:16.021489 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:16.021504 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:16.021586 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:16.021714 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 00:40:16.021808 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":95,"threads_started":1}
I20260807 00:40:16.023135 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:40:16.051874 9432 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:16.052716 9432 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:16.061703 9432 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4730cbed0da643b091c9b4d18e1d1ad8: Generated new TSK 0
I20260807 00:40:16.061932 9432 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 00:40:16.091571 9460 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 00:40:16.091706 9460 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 00:40:16.091912 9424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:16.070909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60178 (local address 127.1.116.190:41767)
0807 00:40:16.070980 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:16.070983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:16.071165 (+ 182us) server_negotiation.cc:407] Connection header received
0807 00:40:16.071256 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:16.071257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:16.071279 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:16.071369 (+ 90us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:40:16.071534 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.072062 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.072934 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.073127 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.073216 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.073422 (+ 206us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:40:16.091659 (+ 18237us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 00:40:16.091830 (+ 171us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60178: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":35}
W20260807 00:40:16.091993 9424 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60178: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 00:40:16.092147 9459 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:16.070817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41767 (local address 127.0.0.1:60178)
0807 00:40:16.071067 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:16.071079 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:16.071182 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:16.071367 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:16.071368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:16.071370 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:40:16.071444 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:16.071446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.072170 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.072172 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:16.072817 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:16.072819 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.073220 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.073221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:16.073283 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.073290 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:40:16.091878 (+ 18588us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 00:40:16.092029 (+ 151us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.116.190:41767: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":127,"threads_started":1}
I20260807 00:40:16.092509 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 9320
2026-08-07T00:40:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (262 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 00:40:16.140008 1490 mini_oidc.cc:152] Starting JWKS server
I20260807 00:40:16.140836 1490 webserver.cc:460] Webserver started at https://127.0.0.1:33217/ using document root <none> and password file <none>
I20260807 00:40:16.140884 1490 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 00:40:16.141050 1490 webserver.cc:460] Webserver started at http://127.0.0.1:38057/ using document root <none> and password file <none>
2026-08-07T00:40:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:16Z Disabled control of system clock
I20260807 00:40:16.150703 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39361
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39361
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33217/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 00:40:16.236961 9471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:16.237169 9471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:16.237192 9471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:16.238345 9471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:16.238389 9471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:16.238402 9471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:16.238415 9471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:16.238428 9471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:40:16.238481 9471 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:40:16.238511 9471 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33217/jwks.json
I20260807 00:40:16.239936 9471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39361
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39361
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33217/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:16.240191 9471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:16.241297 9471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:16.246893 9479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:16.247031 9471 server_base.cc:1034] running on GCE node
W20260807 00:40:16.246929 9478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:16.247198 9482 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:16.247864 9471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:16.248468 9471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:16.249643 9471 hybrid_clock.cc:648] HybridClock initialized: now 1786063216249617 us; error 39 us; skew 500 ppm
I20260807 00:40:16.251613 9471 webserver.cc:460] Webserver started at http://127.1.116.190:36053/ using document root <none> and password file <none>
I20260807 00:40:16.251991 9471 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:16.252264 9471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:16.252506 9471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:16.253787 9471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "88380033df5f45d193c8e77d3ca65bad"
format_stamp: "Formatted at 2026-08-07 00:40:16 on dist-test-slave-h6n0"
server_key: "9a08d829aca3678329b32f2380c9e976"
server_key_iv: "819ec656a95f5390e684dca156308c30"
server_key_version: "encryptionkey@0"
I20260807 00:40:16.254254 9471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "88380033df5f45d193c8e77d3ca65bad"
format_stamp: "Formatted at 2026-08-07 00:40:16 on dist-test-slave-h6n0"
server_key: "9a08d829aca3678329b32f2380c9e976"
server_key_iv: "819ec656a95f5390e684dca156308c30"
server_key_version: "encryptionkey@0"
I20260807 00:40:16.255951 9471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 00:40:16.259019 9490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:16.259299 9471 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260807 00:40:16.259433 9471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "88380033df5f45d193c8e77d3ca65bad"
format_stamp: "Formatted at 2026-08-07 00:40:16 on dist-test-slave-h6n0"
server_key: "9a08d829aca3678329b32f2380c9e976"
server_key_iv: "819ec656a95f5390e684dca156308c30"
server_key_version: "encryptionkey@0"
I20260807 00:40:16.259529 9471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:16.286564 9471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:16.287415 9471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:16.287631 9471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:16.293983 9471 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:39361
I20260807 00:40:16.294104 9565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:39361 every 8 connection(s)
I20260807 00:40:16.294483 9471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:16.295372 9569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:16.296516 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 9471
I20260807 00:40:16.296641 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:16.296829 1490 external_mini_cluster.cc:1428] Setting key b022f20386894da903990509aae3c35c
I20260807 00:40:16.299383 9569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad: Bootstrap starting.
I20260807 00:40:16.300024 9569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:16.300336 9569 log.cc:824] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:16.301275 9569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad: No bootstrap required, opened a new log
I20260807 00:40:16.303613 9569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88380033df5f45d193c8e77d3ca65bad" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39361 } }
I20260807 00:40:16.303754 9569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:16.303778 9569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88380033df5f45d193c8e77d3ca65bad, State: Initialized, Role: FOLLOWER
I20260807 00:40:16.303867 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:16.297486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52122 (local address 127.1.116.190:39361)
0807 00:40:16.297752 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:16.297763 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:16.297795 (+ 32us) server_negotiation.cc:407] Connection header received
0807 00:40:16.298438 (+ 643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:16.298446 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:16.298519 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:16.298721 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:16.298953 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.300316 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.301320 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.301803 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.301914 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.303027 (+ 1113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:16.303032 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:16.303032 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:16.303234 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:16.303279 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:16.303291 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:16.303348 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:16.303464 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 00:40:16.303562 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":119,"threads_started":1}
I20260807 00:40:16.303918 9569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88380033df5f45d193c8e77d3ca65bad" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39361 } }
I20260807 00:40:16.304018 9569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:16.304066 9569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:16.304118 9569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:16.305022 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:40:16.371155 9569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88380033df5f45d193c8e77d3ca65bad" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39361 } }
I20260807 00:40:16.371340 9569 leader_election.cc:302] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88380033df5f45d193c8e77d3ca65bad; no voters:
I20260807 00:40:16.371621 9569 leader_election.cc:288] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:16.371794 9576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:16.371971 9569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:16.372103 9576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [term 1 LEADER]: Becoming Leader. State: Replica: 88380033df5f45d193c8e77d3ca65bad, State: Running, Role: LEADER
I20260807 00:40:16.372251 9576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88380033df5f45d193c8e77d3ca65bad" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39361 } }
I20260807 00:40:16.373603 9577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88380033df5f45d193c8e77d3ca65bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88380033df5f45d193c8e77d3ca65bad" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39361 } } }
I20260807 00:40:16.373736 9577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:16.373764 9579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88380033df5f45d193c8e77d3ca65bad. Latest consensus state: current_term: 1 leader_uuid: "88380033df5f45d193c8e77d3ca65bad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88380033df5f45d193c8e77d3ca65bad" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39361 } } }
I20260807 00:40:16.374073 9579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad [sys.catalog]: This master's current role is: LEADER
W20260807 00:40:16.374650 9595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:40:16.374718 9595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:16.374794 9596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:16.375048 9596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:16.377422 9596 catalog_manager.cc:1347] Generated new cluster ID: 736e70388bd44f748c8c7f9f1d6d5d13
I20260807 00:40:16.377498 9596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:16.394114 9596 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:16.395035 9596 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:16.400516 9596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88380033df5f45d193c8e77d3ca65bad: Generated new TSK 0
I20260807 00:40:16.400763 9596 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:40:16.416718 9574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:16.412694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52128 (local address 127.1.116.190:39361)
0807 00:40:16.412746 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:16.412748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:16.412919 (+ 171us) server_negotiation.cc:407] Connection header received
0807 00:40:16.412988 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:16.412988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:16.412999 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:16.413066 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:40:16.413371 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.413900 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.415206 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.415407 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.415477 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.415712 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:40:16.416495 (+ 783us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33217/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 00:40:16.416639 (+ 144us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33217/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":28}
W20260807 00:40:16.416791 9574 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52128: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33217/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 00:40:16.416868 9605 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:16.412575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39361 (local address 127.0.0.1:52128)
0807 00:40:16.412790 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:16.412805 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:16.412916 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:16.413087 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:16.413088 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:16.413090 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:40:16.413187 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:16.413190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.414004 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.414005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:16.415052 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:16.415056 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.415530 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.415531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:16.415598 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.415607 (+ 9us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:40:16.416605 (+ 998us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33217/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 00:40:16.416739 (+ 134us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.116.190:39361: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33217/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":128,"threads_started":1}
I20260807 00:40:16.417049 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 9471
E20260807 00:40:16.434042 9608 webserver.cc:533] Webserver: sslize failed:
E20260807 00:40:16.434149 9608 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T00:40:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (324 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T00:40:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:16Z Disabled control of system clock
I20260807 00:40:16.476116 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39355
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:40337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39355
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/testchainca.cert with env {}
W20260807 00:40:16.574209 9613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:16.574455 9613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:16.574482 9613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:16.575794 9613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:16.575847 9613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:16.575860 9613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:16.575873 9613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:16.575924 9613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:40:16.575960 9613 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:40:16.575977 9613 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 00:40:16.577682 9613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39355
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:16.578001 9613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:16.578275 9613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:16.585292 9619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:16.585453 9624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:16.585314 9618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:16.585852 9613 server_base.cc:1034] running on GCE node
I20260807 00:40:16.586669 9613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:16.587077 9613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:16.588256 9613 hybrid_clock.cc:648] HybridClock initialized: now 1786063216588232 us; error 39 us; skew 500 ppm
I20260807 00:40:16.589661 9613 webserver.cc:460] Webserver started at http://127.1.116.190:32833/ using document root <none> and password file <none>
I20260807 00:40:16.589956 9613 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:16.590009 9613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:16.590149 9613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:16.591346 9613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "b051553d704343b3920315d07c0fcec2"
format_stamp: "Formatted at 2026-08-07 00:40:16 on dist-test-slave-h6n0"
server_key: "b3e23dcab8e209a19223f83c7361338d"
server_key_iv: "cad64f7740abf469c55119f7cdd691f0"
server_key_version: "encryptionkey@0"
I20260807 00:40:16.591739 9613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "b051553d704343b3920315d07c0fcec2"
format_stamp: "Formatted at 2026-08-07 00:40:16 on dist-test-slave-h6n0"
server_key: "b3e23dcab8e209a19223f83c7361338d"
server_key_iv: "cad64f7740abf469c55119f7cdd691f0"
server_key_version: "encryptionkey@0"
I20260807 00:40:16.593250 9613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:16.594254 9630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:16.594696 9613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:16.594827 9613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "b051553d704343b3920315d07c0fcec2"
format_stamp: "Formatted at 2026-08-07 00:40:16 on dist-test-slave-h6n0"
server_key: "b3e23dcab8e209a19223f83c7361338d"
server_key_iv: "cad64f7740abf469c55119f7cdd691f0"
server_key_version: "encryptionkey@0"
I20260807 00:40:16.594921 9613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:16.617385 9613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:16.617746 9613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:16.617924 9613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:16.622665 9613 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:39355
I20260807 00:40:16.622740 9706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:39355 every 8 connection(s)
I20260807 00:40:16.623121 9613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:16.624467 9707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:16.627517 9707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2: Bootstrap starting.
I20260807 00:40:16.628295 9707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:16.628692 9707 log.cc:824] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:16.629984 9707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2: No bootstrap required, opened a new log
I20260807 00:40:16.631915 9707 raft_consensus.cc:348] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b051553d704343b3920315d07c0fcec2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39355 } }
I20260807 00:40:16.632107 9707 raft_consensus.cc:374] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:16.632131 9707 raft_consensus.cc:729] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b051553d704343b3920315d07c0fcec2, State: Initialized, Role: FOLLOWER
I20260807 00:40:16.632345 9707 consensus_queue.cc:260] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b051553d704343b3920315d07c0fcec2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39355 } }
I20260807 00:40:16.632488 9707 raft_consensus.cc:388] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:16.632527 9707 raft_consensus.cc:482] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:16.632578 9707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:16.632583 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 9613
I20260807 00:40:16.632688 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:16.632932 1490 external_mini_cluster.cc:1428] Setting key 99c817e092c8238bb809d216594b19a7
I20260807 00:40:16.643647 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:16.633565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52068 (local address 127.1.116.190:39355)
0807 00:40:16.633834 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:16.633845 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:16.633884 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:40:16.634001 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:16.634008 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:16.634070 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:16.634242 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:16.634475 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.635505 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.636329 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.636847 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.637004 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.642733 (+ 5729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:16.642745 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:16.642746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:16.642951 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:16.642993 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:16.643007 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:16.643094 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:16.643212 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 00:40:16.643319 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":83,"threads_started":1}
I20260807 00:40:16.645048 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:40:16.686652 9707 raft_consensus.cc:504] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b051553d704343b3920315d07c0fcec2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39355 } }
I20260807 00:40:16.686949 9707 leader_election.cc:302] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b051553d704343b3920315d07c0fcec2; no voters:
I20260807 00:40:16.687325 9707 leader_election.cc:288] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:16.687498 9718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:16.687736 9718 raft_consensus.cc:686] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [term 1 LEADER]: Becoming Leader. State: Replica: b051553d704343b3920315d07c0fcec2, State: Running, Role: LEADER
I20260807 00:40:16.687836 9707 sys_catalog.cc:564] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:16.687949 9718 consensus_queue.cc:237] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b051553d704343b3920315d07c0fcec2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39355 } }
I20260807 00:40:16.688814 9721 sys_catalog.cc:455] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b051553d704343b3920315d07c0fcec2. Latest consensus state: current_term: 1 leader_uuid: "b051553d704343b3920315d07c0fcec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b051553d704343b3920315d07c0fcec2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39355 } } }
I20260807 00:40:16.688911 9721 sys_catalog.cc:458] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:16.689211 9720 sys_catalog.cc:455] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b051553d704343b3920315d07c0fcec2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b051553d704343b3920315d07c0fcec2" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39355 } } }
I20260807 00:40:16.689373 9720 sys_catalog.cc:458] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:16.690029 9727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:16.690698 9727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:16.692618 9727 catalog_manager.cc:1347] Generated new cluster ID: 9be8ed29fbc141d1adc3fcc63a709b60
I20260807 00:40:16.692709 9727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:16.707957 9727 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:16.708791 9727 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:16.712900 9727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b051553d704343b3920315d07c0fcec2: Generated new TSK 0
I20260807 00:40:16.713122 9727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:40:16.891104 9717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:16.730905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.1.116.190:39355)
0807 00:40:16.730979 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:16.730982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:16.731106 (+ 124us) server_negotiation.cc:407] Connection header received
0807 00:40:16.731182 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:16.731183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:16.731198 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:16.731297 (+ 99us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:40:16.731473 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.732281 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.733243 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.733567 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.733653 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.733861 (+ 208us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:40:16.890748 (+156887us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 00:40:16.890981 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52070: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":44}
W20260807 00:40:16.891201 9717 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52070: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 00:40:16.891333 9753 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:16.730802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39355 (local address 127.0.0.1:52070)
0807 00:40:16.731015 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:16.731032 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:16.731103 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:16.731305 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:16.731306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:16.731309 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:40:16.731377 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:16.731380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.732406 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.732407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:16.733116 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:16.733119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:16.733671 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:16.733672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:16.733717 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:16.733742 (+ 25us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:40:16.890981 (+157239us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 00:40:16.891150 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.116.190:39355: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":119,"threads_started":1}
I20260807 00:40:16.891598 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 9613
2026-08-07T00:40:16Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (455 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (98 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (107 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:18 dist-test-slave-h6n0 krb5kdc[9773](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:18 dist-test-slave-h6n0 krb5kdc[9773](info): set up 2 sockets
Aug 07 00:40:18 dist-test-slave-h6n0 krb5kdc[9773](info): commencing operation
krb5kdc: starting...
W20260807 00:40:19.536229 1490 mini_kdc.cc:121] Time spent starting KDC: real 2.409s user 0.000s sys 0.012s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:19 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:19.667317 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38899
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:38899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:19.763074 9789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:19.763306 9789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:19.763331 9789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:19.764536 9789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:19.764582 9789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:19.764597 9789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:19.764611 9789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:19.764622 9789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:19.766203 9789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:38899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:19.766481 9789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:19.766775 9789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:19.772050 9801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:19.772095 9798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:19.772116 9799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:19.772367 9789 server_base.cc:1034] running on GCE node
I20260807 00:40:19.772667 9789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:19.773047 9789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:19.774235 9789 hybrid_clock.cc:648] HybridClock initialized: now 1786063219774227 us; error 61 us; skew 500 ppm
Aug 07 00:40:19 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:19.776975 9789 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:19.777431 9789 webserver.cc:460] Webserver started at http://127.1.116.190:34865/ using document root <none> and password file <none>
I20260807 00:40:19.777714 9789 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:19.777786 9789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:19.777940 9789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:19.779206 9789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "5bbb3a4c7c924684811b51a0d0705e97"
format_stamp: "Formatted at 2026-08-07 00:40:19 on dist-test-slave-h6n0"
server_key: "54f070b332b9d23032e6b1c4d6af4400"
server_key_iv: "4ef4720446c9b8bd7a6f2da367364746"
server_key_version: "encryptionkey@0"
I20260807 00:40:19.779595 9789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "5bbb3a4c7c924684811b51a0d0705e97"
format_stamp: "Formatted at 2026-08-07 00:40:19 on dist-test-slave-h6n0"
server_key: "54f070b332b9d23032e6b1c4d6af4400"
server_key_iv: "4ef4720446c9b8bd7a6f2da367364746"
server_key_version: "encryptionkey@0"
I20260807 00:40:19.781081 9789 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 00:40:19.781955 9808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:19.782202 9789 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:19.782325 9789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "5bbb3a4c7c924684811b51a0d0705e97"
format_stamp: "Formatted at 2026-08-07 00:40:19 on dist-test-slave-h6n0"
server_key: "54f070b332b9d23032e6b1c4d6af4400"
server_key_iv: "4ef4720446c9b8bd7a6f2da367364746"
server_key_version: "encryptionkey@0"
I20260807 00:40:19.782400 9789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:19.802295 9789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:19.804669 9789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:19.804834 9789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:19.809374 9789 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:38899
I20260807 00:40:19.809448 9878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:38899 every 8 connection(s)
I20260807 00:40:19.809814 9789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:19.810992 9879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:19.813526 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 9789
I20260807 00:40:19.813647 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:19.813876 1490 external_mini_cluster.cc:1428] Setting key 7eda5a991893f81a18cc9beefc856e2a
I20260807 00:40:19.814006 9879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97: Bootstrap starting.
I20260807 00:40:19.814940 9879 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:19.815307 9879 log.cc:824] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:19.816089 9879 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97: No bootstrap required, opened a new log
I20260807 00:40:19.818159 9879 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbb3a4c7c924684811b51a0d0705e97" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38899 } }
I20260807 00:40:19.818346 9879 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:19.818387 9879 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bbb3a4c7c924684811b51a0d0705e97, State: Initialized, Role: FOLLOWER
I20260807 00:40:19.818605 9879 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbb3a4c7c924684811b51a0d0705e97" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38899 } }
I20260807 00:40:19.818723 9879 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:19.818787 9879 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:19.818850 9879 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:19.819495 9879 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbb3a4c7c924684811b51a0d0705e97" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38899 } }
I20260807 00:40:19.819625 9879 leader_election.cc:302] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bbb3a4c7c924684811b51a0d0705e97; no voters:
I20260807 00:40:19.819866 9879 leader_election.cc:288] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:19.819895 9886 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:19.820047 9886 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [term 1 LEADER]: Becoming Leader. State: Replica: 5bbb3a4c7c924684811b51a0d0705e97, State: Running, Role: LEADER
I20260807 00:40:19.820209 9879 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:19.820211 9886 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbb3a4c7c924684811b51a0d0705e97" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38899 } }
I20260807 00:40:19.820783 9887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bbb3a4c7c924684811b51a0d0705e97. Latest consensus state: current_term: 1 leader_uuid: "5bbb3a4c7c924684811b51a0d0705e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbb3a4c7c924684811b51a0d0705e97" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38899 } } }
Aug 07 00:40:19 dist-test-slave-h6n0 krb5kdc[9773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:19.820812 9888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bbb3a4c7c924684811b51a0d0705e97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bbb3a4c7c924684811b51a0d0705e97" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38899 } } }
I20260807 00:40:19.821030 9888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:19.821339 9899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:19.820931 9887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:19.822258 9899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:19.825461 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:19.814521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53944 (local address 127.1.116.190:38899)
0807 00:40:19.814749 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:19.814757 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:19.814785 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:40:19.815458 (+ 673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:19.815465 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:19.815514 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:19.815678 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:19.816407 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:19.817294 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:19.818069 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:19.818600 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:19.818715 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:19.821461 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:19.821465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:19.821465 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:19.821477 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:19.823526 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:19.823944 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:19.823949 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:19.823949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:19.824091 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:19.824338 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:19.824339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:19.824339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:19.824689 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:19.824877 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:19.825040 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 00:40:19.825229 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260807 00:40:19.827072 9899 catalog_manager.cc:1347] Generated new cluster ID: 47ce59f3177f4c7e8a691bae954abdaf
I20260807 00:40:19.827148 9899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:19.842422 9899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:19.843284 9899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:19.847056 9899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bbb3a4c7c924684811b51a0d0705e97: Generated new TSK 0
I20260807 00:40:19.847260 9899 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:40:19.882316 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38899
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:19.967794 9922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:19.968106 9922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:19.968222 9922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:19.969381 9922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:19.969447 9922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:19.969496 9922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:19.971131 9922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:19.971407 9922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:19.971644 9922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:19.976778 9934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:19.976824 9936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:19.976778 9933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:19.977044 9922 server_base.cc:1034] running on GCE node
I20260807 00:40:19.977339 9922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:19.977612 9922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:19.978785 9922 hybrid_clock.cc:648] HybridClock initialized: now 1786063219978775 us; error 31 us; skew 500 ppm
Aug 07 00:40:19 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:19.981608 9922 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:40:19.982177 9922 webserver.cc:460] Webserver started at http://127.1.116.129:38001/ using document root <none> and password file <none>
I20260807 00:40:19.982431 9922 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:19.982502 9922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:19.982723 9922 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:19.983884 9922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "27447200b1914ce4ad00a31491ae1a88"
format_stamp: "Formatted at 2026-08-07 00:40:19 on dist-test-slave-h6n0"
server_key: "e07ddb8e4a4df39139fb4658f613110f"
server_key_iv: "31829968965dcf719ecd66bfcf3ce10c"
server_key_version: "encryptionkey@0"
I20260807 00:40:19.984289 9922 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "27447200b1914ce4ad00a31491ae1a88"
format_stamp: "Formatted at 2026-08-07 00:40:19 on dist-test-slave-h6n0"
server_key: "e07ddb8e4a4df39139fb4658f613110f"
server_key_iv: "31829968965dcf719ecd66bfcf3ce10c"
server_key_version: "encryptionkey@0"
I20260807 00:40:19.986023 9922 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 00:40:19.986980 9943 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:19.987211 9922 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:19.987329 9922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "27447200b1914ce4ad00a31491ae1a88"
format_stamp: "Formatted at 2026-08-07 00:40:19 on dist-test-slave-h6n0"
server_key: "e07ddb8e4a4df39139fb4658f613110f"
server_key_iv: "31829968965dcf719ecd66bfcf3ce10c"
server_key_version: "encryptionkey@0"
I20260807 00:40:19.987416 9922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:19.999832 9922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:20.001909 9922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:20.002085 9922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:20.002357 9922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:20.002826 9922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:20.002885 9922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.002952 9922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:20.002982 9922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.010649 9922 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:42333
I20260807 00:40:20.010751 10106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:42333 every 8 connection(s)
I20260807 00:40:20.011101 9922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:20.019470 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 9922
I20260807 00:40:20.019620 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:20.019873 1490 external_mini_cluster.cc:1428] Setting key ca57f1a46067d9bb13d16c72dc393b25
I20260807 00:40:20.021911 10110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.011895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38899 (local address 127.1.116.129:44287)
0807 00:40:20.012069 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:20.012110 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:20.013135 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:20.013425 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:20.013429 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:20.013707 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:20.013928 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:20.013932 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.014826 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.014826 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:20.015697 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:20.015699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.016127 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.016128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:20.016253 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.016725 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:20.016734 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:20.018280 (+ 1546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:20.020122 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:20.020130 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:20.020133 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:20.020502 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:20.020817 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:20.020818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:20.020819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:20.020954 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:20.021282 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:20.021288 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:20.021464 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 00:40:20.021584 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 00:40:20.021749 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260807 00:40:20.021953 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.011915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:44287 (local address 127.1.116.190:38899)
0807 00:40:20.011974 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.011976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.012238 (+ 262us) server_negotiation.cc:407] Connection header received
0807 00:40:20.013283 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.013284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.013297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.013362 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.014114 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.014736 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.015782 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.015992 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.016048 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.018375 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:20.018377 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:20.018377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:20.018384 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:20.020051 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.020602 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.020604 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.020605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.020717 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.021042 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.021043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.021043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.021213 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:20.021293 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:20.021638 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 00:40:20.021793 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:40:20.022820 10108 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38899
I20260807 00:40:20.022939 10108 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:20.023140 10108 heartbeater.cc:511] Master 127.1.116.190:38899 requested a full tablet report, sending...
I20260807 00:40:20.023690 9830 ts_manager.cc:194] Registered new tserver with Master: 27447200b1914ce4ad00a31491ae1a88 (127.1.116.129:42333)
I20260807 00:40:20.024302 9830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:44287
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:40:20.048588 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38899
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:20.139336 10116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:20.139575 10116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:20.139600 10116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:20.140799 10116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:20.140847 10116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:20.140875 10116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:20.142411 10116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:20.142719 10116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:20.143004 10116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:20.148552 10124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:20.148515 10127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:20.148519 10125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:20.148900 10116 server_base.cc:1034] running on GCE node
I20260807 00:40:20.149201 10116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:20.149482 10116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:20.150594 10116 hybrid_clock.cc:648] HybridClock initialized: now 1786063220150574 us; error 38 us; skew 500 ppm
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.153506 10116 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:40:20.153956 10116 webserver.cc:460] Webserver started at http://127.1.116.130:46165/ using document root <none> and password file <none>
I20260807 00:40:20.154258 10116 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:20.154309 10116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:20.154407 10116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:20.155548 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "1012ce7ddad549ce96e2704aac5de598"
format_stamp: "Formatted at 2026-08-07 00:40:20 on dist-test-slave-h6n0"
server_key: "e1e2210503df11f0b00dd6ffc3fca26d"
server_key_iv: "2c2daf70dd4af15fa1f8f53fdfb887d0"
server_key_version: "encryptionkey@0"
I20260807 00:40:20.155866 10116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "1012ce7ddad549ce96e2704aac5de598"
format_stamp: "Formatted at 2026-08-07 00:40:20 on dist-test-slave-h6n0"
server_key: "e1e2210503df11f0b00dd6ffc3fca26d"
server_key_iv: "2c2daf70dd4af15fa1f8f53fdfb887d0"
server_key_version: "encryptionkey@0"
I20260807 00:40:20.157270 10116 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:20.158063 10142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.158249 10116 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:20.158353 10116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "1012ce7ddad549ce96e2704aac5de598"
format_stamp: "Formatted at 2026-08-07 00:40:20 on dist-test-slave-h6n0"
server_key: "e1e2210503df11f0b00dd6ffc3fca26d"
server_key_iv: "2c2daf70dd4af15fa1f8f53fdfb887d0"
server_key_version: "encryptionkey@0"
I20260807 00:40:20.158430 10116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:20.173568 10116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:20.175663 10116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:20.175783 10116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:20.175998 10116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:20.176294 10116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:20.176329 10116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.176352 10116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:20.176366 10116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.183537 10116 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:33493
I20260807 00:40:20.183630 10297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:33493 every 8 connection(s)
I20260807 00:40:20.183972 10116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:20.184561 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 10116
I20260807 00:40:20.184679 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:20.184908 1490 external_mini_cluster.cc:1428] Setting key cbc80b2f29f53bda9a27fcd5e9d68847
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:20.196215 10302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.185449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38899 (local address 127.1.116.130:40319)
0807 00:40:20.185640 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:20.185680 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:20.187047 (+ 1367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:20.187332 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:20.187336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:20.187738 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:20.188014 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:20.188020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.189097 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.189098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:20.189904 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:20.189906 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.190441 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.190442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:20.190633 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.191133 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:20.191142 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:20.192651 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:20.194726 (+ 2075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:20.194733 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:20.194735 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:20.194997 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:20.195260 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:20.195261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:20.195261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:20.195374 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:20.195725 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:20.195730 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:20.195841 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 00:40:20.195932 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 00:40:20.196053 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":121,"threads_started":1}
I20260807 00:40:20.196221 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.185473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:40319 (local address 127.1.116.190:38899)
0807 00:40:20.185549 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.185551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.185783 (+ 232us) server_negotiation.cc:407] Connection header received
0807 00:40:20.187165 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.187166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.187177 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.187243 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.188143 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.188990 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.190002 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.190322 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.190405 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.192754 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:20.192757 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:20.192757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:20.192768 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:20.194628 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.195095 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.195096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.195097 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.195167 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.195459 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.195461 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.195461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.195634 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:20.195738 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:20.195992 (+ 254us) server_negotiation.cc:299] Negotiation successful
0807 00:40:20.196097 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
I20260807 00:40:20.196879 10300 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38899
I20260807 00:40:20.197027 10300 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:20.197290 10300 heartbeater.cc:511] Master 127.1.116.190:38899 requested a full tablet report, sending...
I20260807 00:40:20.197767 9830 ts_manager.cc:194] Registered new tserver with Master: 1012ce7ddad549ce96e2704aac5de598 (127.1.116.130:33493)
I20260807 00:40:20.198177 9830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:40319
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:40:20.214673 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38899
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:20.305540 10307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:20.305768 10307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:20.305806 10307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:20.306946 10307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:20.306989 10307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:20.307015 10307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:20.308420 10307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:20.308617 10307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:20.308782 10307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:40:20.313939 10307 server_base.cc:1034] running on GCE node
W20260807 00:40:20.313936 10313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:20.313961 10312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:20.313936 10315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:20.314639 10307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:20.314998 10307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:20.316197 10307 hybrid_clock.cc:648] HybridClock initialized: now 1786063220316176 us; error 42 us; skew 500 ppm
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.318961 10307 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:40:20.319453 10307 webserver.cc:460] Webserver started at http://127.1.116.131:43347/ using document root <none> and password file <none>
I20260807 00:40:20.319722 10307 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:20.319794 10307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:20.319949 10307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:20.321182 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "0df6ac080d7f43f6af650426e1e4a4a5"
format_stamp: "Formatted at 2026-08-07 00:40:20 on dist-test-slave-h6n0"
server_key: "e382fba8e99472bcca85610f3e24e6fb"
server_key_iv: "bf5805b568ea3ac0b05ad78f352b804f"
server_key_version: "encryptionkey@0"
I20260807 00:40:20.321561 10307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "0df6ac080d7f43f6af650426e1e4a4a5"
format_stamp: "Formatted at 2026-08-07 00:40:20 on dist-test-slave-h6n0"
server_key: "e382fba8e99472bcca85610f3e24e6fb"
server_key_iv: "bf5805b568ea3ac0b05ad78f352b804f"
server_key_version: "encryptionkey@0"
I20260807 00:40:20.323089 10307 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:20.323953 10325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.324237 10307 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:20.324324 10307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "0df6ac080d7f43f6af650426e1e4a4a5"
format_stamp: "Formatted at 2026-08-07 00:40:20 on dist-test-slave-h6n0"
server_key: "e382fba8e99472bcca85610f3e24e6fb"
server_key_iv: "bf5805b568ea3ac0b05ad78f352b804f"
server_key_version: "encryptionkey@0"
I20260807 00:40:20.324420 10307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:20.340579 10307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:20.342633 10307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:20.342779 10307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:20.343008 10307 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:20.343318 10307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:20.343357 10307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.343377 10307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:20.343431 10307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:20.350274 10307 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:42733
I20260807 00:40:20.350329 10479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:42733 every 8 connection(s)
I20260807 00:40:20.350688 10307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:20.360904 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 10307
I20260807 00:40:20.361029 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:20.361176 9883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.351504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:52793 (local address 127.1.116.190:38899)
0807 00:40:20.351624 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.351627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.351834 (+ 207us) server_negotiation.cc:407] Connection header received
0807 00:40:20.352940 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.352942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.352955 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.353028 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.353735 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.354364 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.355456 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.355686 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.355778 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.358101 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:20.358103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:20.358104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:20.358112 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:20.359511 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.359963 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.359965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.359965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.360031 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.360305 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.360306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.360307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.360434 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:20.360518 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:20.360943 (+ 425us) server_negotiation.cc:299] Negotiation successful
0807 00:40:20.361088 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260807 00:40:20.361266 10483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.351463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38899 (local address 127.1.116.131:52793)
0807 00:40:20.351681 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:20.351731 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:20.352816 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:20.353107 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:20.353112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:20.353413 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:20.353636 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:20.353640 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.354459 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.354460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:20.355358 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:20.355362 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.355791 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.355792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:20.355926 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.356458 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:20.356467 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:20.357993 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:20.359616 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:20.359623 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:20.359625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:20.359877 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:20.360112 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:20.360113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:20.360113 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:20.360233 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:20.360509 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:20.360516 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:20.360661 (+ 145us) client_negotiation.cc:769] Sending connection context
0807 00:40:20.360806 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 00:40:20.360995 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":88,"threads_started":1}
I20260807 00:40:20.361279 1490 external_mini_cluster.cc:1428] Setting key c9a8d182c3be5896e0af4b25140eccd1
I20260807 00:40:20.361889 10480 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38899
I20260807 00:40:20.362020 10480 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:20.362227 10480 heartbeater.cc:511] Master 127.1.116.190:38899 requested a full tablet report, sending...
I20260807 00:40:20.362739 9829 ts_manager.cc:194] Registered new tserver with Master: 0df6ac080d7f43f6af650426e1e4a4a5 (127.1.116.131:42733)
I20260807 00:40:20.363153 9829 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:52793
I20260807 00:40:20.373420 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/test-admin.keytab.
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.391116 1490 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 00:40:20.393263 1490 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc)
W20260807 00:40:20.393285 10490 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:20.392165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54776 (local address 127.1.116.129:42333)
0807 00:40:20.392419 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.392423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.392435 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:20.392460 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.392462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.392476 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.392566 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.393085 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54776: BlockingRecv error: recv got EOF from 127.0.0.1:54776 (error 108)
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":147,"threads_started":1}
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.394470 1490 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.395565 1490 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:40:20.404106 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.396214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54782 (local address 127.1.116.129:42333)
0807 00:40:20.396296 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.396297 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.396310 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:20.396378 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.396378 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.396389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.396455 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.396915 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.397655 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.398500 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.398888 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.398952 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.401241 (+ 2289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:20.401248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:20.401249 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:20.401255 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:20.402974 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.403330 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.403332 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.403332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.403385 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.403587 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.403588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.403588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.403709 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:20.403824 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:20.403911 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 00:40:20.403975 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 00:40:20.406472 10490 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:20.405725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54794 (local address 127.1.116.129:42333)
0807 00:40:20.405809 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.405812 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.405900 (+ 88us) server_negotiation.cc:407] Connection header received
0807 00:40:20.405996 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.405998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.406012 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.406084 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.406411 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54794: BlockingRecv error: recv got EOF from 127.0.0.1:54794 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 00:40:20.406616 1490 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc' of type FILE
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.407676 1490 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.408718 1490 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:40:20.416741 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.409280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.1.116.129:42333)
0807 00:40:20.409360 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.409363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.409399 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:40:20.409498 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.409499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.409509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.409571 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.409996 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.410704 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.411487 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.411740 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.411835 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.413955 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:20.413956 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:20.413957 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:20.413965 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:20.415426 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.415823 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.415825 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.415825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.415889 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.416117 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.416118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.416119 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.416236 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:20.416355 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:20.416529 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 00:40:20.416621 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 00:40:20.418717 10490 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:20.417886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54810 (local address 127.1.116.129:42333)
0807 00:40:20.417960 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.417962 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.418029 (+ 67us) server_negotiation.cc:407] Connection header received
0807 00:40:20.418108 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.418109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.418119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.418169 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.418673 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54810: BlockingRecv error: recv got EOF from 127.0.0.1:54810 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 00:40:20.418999 1490 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786063190385254-1490-0/krb5kdc/krb5cc' of type FILE
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.420166 1490 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:20.421322 1490 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[9773](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063220, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:40:20.429584 10490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:20.421923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54826 (local address 127.1.116.129:42333)
0807 00:40:20.421997 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:20.421999 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:20.422027 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:40:20.422093 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:20.422094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:20.422103 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:20.422153 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:20.422673 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.423196 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.423972 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:20.424206 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:20.424293 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:20.426489 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:20.426490 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:20.426491 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:20.426498 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:20.428325 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.428724 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.428725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.428726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.428791 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:20.429025 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:20.429026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:20.429026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:20.429138 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:20.429268 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:20.429385 (+ 117us) server_negotiation.cc:299] Negotiation successful
0807 00:40:20.429469 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:40:20.430646 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 9922
I20260807 00:40:20.445672 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 10116
I20260807 00:40:20.459036 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 10307
I20260807 00:40:20.476840 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 9789
2026-08-07T00:40:20Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3379 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[10507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[10507](info): set up 2 sockets
Aug 07 00:40:20 dist-test-slave-h6n0 krb5kdc[10507](info): commencing operation
krb5kdc: starting...
W20260807 00:40:21.730872 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.003s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:21 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:21Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab.
Entry for principal oryx/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab.
I20260807 00:40:21.861354 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:40717
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab
--principal=oryx/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:40717
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:21.942392 10525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:21.942617 10525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:21.942641 10525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:21.943774 10525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:21.943818 10525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:21.943832 10525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:21.943846 10525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:21.943907 10525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:21.945441 10525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:40717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab
--principal=oryx/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:40717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:21.945698 10525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:21.945941 10525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:21.951141 10531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:21.951210 10532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:21.951200 10535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:21.951596 10525 server_base.cc:1034] running on GCE node
I20260807 00:40:21.951880 10525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:21.952205 10525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:21.953461 10525 hybrid_clock.cc:648] HybridClock initialized: now 1786063221953435 us; error 56 us; skew 500 ppm
Aug 07 00:40:21 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063221, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:21.959251 10525 init.cc:377] Logged in from keytab as oryx/127.1.116.190@KRBTEST.COM (short username oryx)
I20260807 00:40:21.959750 10525 webserver.cc:460] Webserver started at http://127.1.116.190:38875/ using document root <none> and password file <none>
I20260807 00:40:21.959998 10525 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:21.960048 10525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:21.960213 10525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:21.961266 10525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "c01d75845cb043868f37d813b9b698dc"
format_stamp: "Formatted at 2026-08-07 00:40:21 on dist-test-slave-h6n0"
server_key: "bd9781273708fab5a515ab3c401278c3"
server_key_iv: "6dd03d36720939d543d6f91f77e29158"
server_key_version: "encryptionkey@0"
I20260807 00:40:21.961629 10525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "c01d75845cb043868f37d813b9b698dc"
format_stamp: "Formatted at 2026-08-07 00:40:21 on dist-test-slave-h6n0"
server_key: "bd9781273708fab5a515ab3c401278c3"
server_key_iv: "6dd03d36720939d543d6f91f77e29158"
server_key_version: "encryptionkey@0"
I20260807 00:40:21.963126 10525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:21.964052 10546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:21.964319 10525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:21.964432 10525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "c01d75845cb043868f37d813b9b698dc"
format_stamp: "Formatted at 2026-08-07 00:40:21 on dist-test-slave-h6n0"
server_key: "bd9781273708fab5a515ab3c401278c3"
server_key_iv: "6dd03d36720939d543d6f91f77e29158"
server_key_version: "encryptionkey@0"
I20260807 00:40:21.964519 10525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:21.993868 10525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:21.995677 10525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:21.995839 10525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:22.000514 10525 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:40717
I20260807 00:40:22.000586 10621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:40717 every 8 connection(s)
I20260807 00:40:22.000941 10525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:22.002095 10622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:22.004823 10622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: Bootstrap starting.
I20260807 00:40:22.005447 10622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:22.005729 10622 log.cc:824] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:22.006441 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 10525
I20260807 00:40:22.006618 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:22.006665 10622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: No bootstrap required, opened a new log
I20260807 00:40:22.006899 1490 external_mini_cluster.cc:1428] Setting key 97bdab0d1d22d09f8f3f81166a3852e9
I20260807 00:40:22.009549 10622 raft_consensus.cc:348] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } }
I20260807 00:40:22.009703 10622 raft_consensus.cc:374] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:22.009727 10622 raft_consensus.cc:729] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c01d75845cb043868f37d813b9b698dc, State: Initialized, Role: FOLLOWER
I20260807 00:40:22.009871 10622 consensus_queue.cc:260] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } }
I20260807 00:40:22.009953 10622 raft_consensus.cc:388] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:22.010010 10622 raft_consensus.cc:482] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:22.010066 10622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:22.010874 10622 raft_consensus.cc:504] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } }
I20260807 00:40:22.011011 10622 leader_election.cc:302] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c01d75845cb043868f37d813b9b698dc; no voters:
I20260807 00:40:22.011214 10622 leader_election.cc:288] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:22.011297 10630 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:22.011552 10630 raft_consensus.cc:686] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 1 LEADER]: Becoming Leader. State: Replica: c01d75845cb043868f37d813b9b698dc, State: Running, Role: LEADER
I20260807 00:40:22.011560 10622 sys_catalog.cc:564] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:22.011737 10630 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } }
I20260807 00:40:22.013334 10631 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } }
I20260807 00:40:22.013440 10633 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01d75845cb043868f37d813b9b698dc. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } }
I20260807 00:40:22.013778 10633 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
W20260807 00:40:22.014183 10654 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:40:22.014257 10654 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:22.014389 10658 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:22.014607 10658 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:22.013458 10631 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:22.016480 10658 catalog_manager.cc:1347] Generated new cluster ID: e4574c42febf434f812f8750e8f37439
I20260807 00:40:22.016542 10658 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:22.020380 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.007499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60926 (local address 127.1.116.190:40717)
0807 00:40:22.007728 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.007736 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.007761 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:40:22.008336 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.008342 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.008387 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.008507 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.009416 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.010870 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.012009 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.012602 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.012716 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.016125 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.016130 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.016131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.016144 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.018402 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.018902 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.018905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.018906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.019053 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.019307 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.019308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.019308 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.019622 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.019806 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.019985 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.020140 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":112,"threads_started":1}
I20260807 00:40:22.043663 10658 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:22.044271 10658 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:22.049402 10658 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: Generated new TSK 0
I20260807 00:40:22.049631 10658 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab.
Entry for principal oryx/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab.
I20260807 00:40:22.081822 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40717
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab
--principal=oryx/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:22.173368 10663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:22.173604 10663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:22.173627 10663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:22.174834 10663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:22.174876 10663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:22.174903 10663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:22.176450 10663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab
--principal=oryx/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:22.176724 10663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:22.176995 10663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:22.182500 10673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.182595 10670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.182499 10671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:22.182817 10663 server_base.cc:1034] running on GCE node
I20260807 00:40:22.183130 10663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:22.183476 10663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:22.184639 10663 hybrid_clock.cc:648] HybridClock initialized: now 1786063222184620 us; error 44 us; skew 500 ppm
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:22.187479 10663 init.cc:377] Logged in from keytab as oryx/127.1.116.129@KRBTEST.COM (short username oryx)
I20260807 00:40:22.187938 10663 webserver.cc:460] Webserver started at http://127.1.116.129:43629/ using document root <none> and password file <none>
I20260807 00:40:22.188205 10663 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:22.188277 10663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:22.188436 10663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:22.189546 10663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "f3b1ba619ef64d509ea97bf5661c9799"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "f2d6a47ecb5d80f91bac2693b21716bd"
server_key_iv: "027878602ee0d73715a89a15fd014696"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.189927 10663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "f3b1ba619ef64d509ea97bf5661c9799"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "f2d6a47ecb5d80f91bac2693b21716bd"
server_key_iv: "027878602ee0d73715a89a15fd014696"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.191510 10663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:22.192322 10683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.192554 10663 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:22.192674 10663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "f3b1ba619ef64d509ea97bf5661c9799"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "f2d6a47ecb5d80f91bac2693b21716bd"
server_key_iv: "027878602ee0d73715a89a15fd014696"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.192751 10663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:22.213548 10663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:22.215843 10663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:22.216005 10663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:22.216230 10663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:22.216540 10663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:22.216598 10663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.216640 10663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:22.216670 10663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.223745 10663 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:37929
I20260807 00:40:22.223898 10856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:37929 every 8 connection(s)
I20260807 00:40:22.224197 10663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:22.228463 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 10663
I20260807 00:40:22.228585 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:22.228971 1490 external_mini_cluster.cc:1428] Setting key d8fc8e54e177aad331860cb9983d3c97
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:22.237071 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.225385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:52915 (local address 127.1.116.190:40717)
0807 00:40:22.225477 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.225479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.225736 (+ 257us) server_negotiation.cc:407] Connection header received
0807 00:40:22.226334 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.226335 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.226348 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.226421 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.227492 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.228125 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.229939 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.230292 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.230424 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.233514 (+ 3090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.233516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.233516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.233525 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.235449 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.235952 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.235953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.235954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.236027 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.236284 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.236284 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.236285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.236450 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.236537 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.236793 (+ 256us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.236913 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:40:22.237121 10859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.225265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.129:52915)
0807 00:40:22.225557 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.225605 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.226195 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.226480 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.226484 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.227172 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.227386 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.227392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.228270 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.228272 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.229715 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.229719 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.230454 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.230455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.230667 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.231428 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.231440 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.233383 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.235556 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.235563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.235564 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.235847 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.236110 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.236110 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.236110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.236209 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.236536 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.236541 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.236650 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.236733 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.236909 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":122,"threads_started":1}
I20260807 00:40:22.238217 10857 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40717
I20260807 00:40:22.238371 10857 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:22.238670 10857 heartbeater.cc:511] Master 127.1.116.190:40717 requested a full tablet report, sending...
I20260807 00:40:22.239450 10570 ts_manager.cc:194] Registered new tserver with Master: f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129:37929)
I20260807 00:40:22.240481 10570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:52915
WARNING: no policy specified for oryx/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab.
Entry for principal oryx/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab.
I20260807 00:40:22.261157 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40717
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab
--principal=oryx/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:22.356768 10862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:22.357049 10862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:22.357095 10862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:22.358263 10862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:22.358305 10862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:22.358331 10862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:22.359808 10862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab
--principal=oryx/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:22.360037 10862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:22.360235 10862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:40:22.366255 10862 server_base.cc:1034] running on GCE node
W20260807 00:40:22.366137 10872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.366119 10869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.366119 10870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:22.366750 10862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:22.367065 10862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:22.368232 10862 hybrid_clock.cc:648] HybridClock initialized: now 1786063222368206 us; error 43 us; skew 500 ppm
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:22.371129 10862 init.cc:377] Logged in from keytab as oryx/127.1.116.130@KRBTEST.COM (short username oryx)
I20260807 00:40:22.371626 10862 webserver.cc:460] Webserver started at http://127.1.116.130:45279/ using document root <none> and password file <none>
I20260807 00:40:22.371887 10862 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:22.371968 10862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:22.372121 10862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:22.373217 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "e263fdb0d6a848f986ebcce331fc9c45"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "73c7c6fa6b1cf8d8c00f67ad124b88cc"
server_key_iv: "e88b12953882878fc8593aa4b2f3307d"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.373596 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "e263fdb0d6a848f986ebcce331fc9c45"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "73c7c6fa6b1cf8d8c00f67ad124b88cc"
server_key_iv: "e88b12953882878fc8593aa4b2f3307d"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.375135 10862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:22.376031 10880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.376286 10862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:22.376413 10862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "e263fdb0d6a848f986ebcce331fc9c45"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "73c7c6fa6b1cf8d8c00f67ad124b88cc"
server_key_iv: "e88b12953882878fc8593aa4b2f3307d"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.376497 10862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:22.388026 10862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:22.390239 10862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:22.390401 10862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:22.390678 10862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:22.390993 10862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:22.391049 10862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.391083 10862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:22.391110 10862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.398059 10862 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:37965
I20260807 00:40:22.398205 11041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:37965 every 8 connection(s)
I20260807 00:40:22.398487 10862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:22.407912 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 10862
I20260807 00:40:22.408021 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:22.408299 1490 external_mini_cluster.cc:1428] Setting key 59edecd04136d2f2ea254d873861a2e6
I20260807 00:40:22.408365 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.399424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:38961 (local address 127.1.116.190:40717)
0807 00:40:22.399496 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.399498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.399722 (+ 224us) server_negotiation.cc:407] Connection header received
0807 00:40:22.400258 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.400259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.400273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.400325 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.401235 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.401757 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.402761 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.402976 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.403054 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.405390 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.405391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.405392 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.405400 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.406910 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.407308 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.407309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.407309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.407371 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.407637 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.407638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.407639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.407755 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.407830 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.408187 (+ 357us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.408273 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:40:22.408542 11047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.399386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.130:38961)
0807 00:40:22.399595 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.399633 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.400147 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.400412 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.400416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.400958 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.401133 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.401137 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.401853 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.401854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.402667 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.402670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.403061 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.403063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.403199 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.403685 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.403692 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.405281 (+ 1589us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.407002 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.407008 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.407011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.407228 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.407440 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.407441 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.407442 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.407559 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.407832 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.407839 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.407996 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.408158 (+ 162us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.408312 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":109,"threads_started":1}
I20260807 00:40:22.409180 11043 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40717
I20260807 00:40:22.409312 11043 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:22.409516 11043 heartbeater.cc:511] Master 127.1.116.190:40717 requested a full tablet report, sending...
I20260807 00:40:22.409992 10570 ts_manager.cc:194] Registered new tserver with Master: e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965)
I20260807 00:40:22.410419 10570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:38961
WARNING: no policy specified for oryx/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab.
Entry for principal oryx/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab.
I20260807 00:40:22.437620 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40717
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab
--principal=oryx/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:22.531378 11050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:22.531618 11050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:22.531651 11050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:22.532840 11050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:22.532886 11050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:22.532917 11050 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:22.534479 11050 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab
--principal=oryx/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:40717
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:22.534806 11050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:22.535086 11050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:22.540820 11058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.540740 11057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.540740 11060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:22.540832 11050 server_base.cc:1034] running on GCE node
I20260807 00:40:22.541234 11050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:22.541560 11050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:22.542732 11050 hybrid_clock.cc:648] HybridClock initialized: now 1786063222542713 us; error 35 us; skew 500 ppm
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:22.545523 11050 init.cc:377] Logged in from keytab as oryx/127.1.116.131@KRBTEST.COM (short username oryx)
I20260807 00:40:22.546006 11050 webserver.cc:460] Webserver started at http://127.1.116.131:38367/ using document root <none> and password file <none>
I20260807 00:40:22.546275 11050 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:22.546346 11050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:22.546504 11050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:22.547662 11050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "90eb9bf8a88d4e22974c0a9513b6241a"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "7ec5c8edb5da1efda9588e521d3dd9d0"
server_key_iv: "1893234c2ad21d6da4c77ac8d387240b"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.548092 11050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "90eb9bf8a88d4e22974c0a9513b6241a"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "7ec5c8edb5da1efda9588e521d3dd9d0"
server_key_iv: "1893234c2ad21d6da4c77ac8d387240b"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.549592 11050 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:22.550657 11069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.550868 11050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:22.550980 11050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "90eb9bf8a88d4e22974c0a9513b6241a"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "7ec5c8edb5da1efda9588e521d3dd9d0"
server_key_iv: "1893234c2ad21d6da4c77ac8d387240b"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.551082 11050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:22.558585 11050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:22.560863 11050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:22.561024 11050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:22.561311 11050 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:22.561694 11050 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:22.561753 11050 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.561798 11050 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:22.561825 11050 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.569258 11050 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:37721
I20260807 00:40:22.569350 11236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:37721 every 8 connection(s)
I20260807 00:40:22.569722 11050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:22.573742 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 11050
I20260807 00:40:22.573843 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:22.574031 1490 external_mini_cluster.cc:1428] Setting key 54efe2c79ff034d78372a4783717f3fa
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:22.579892 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.570631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:50909 (local address 127.1.116.190:40717)
0807 00:40:22.570724 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.570727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.570962 (+ 235us) server_negotiation.cc:407] Connection header received
0807 00:40:22.571449 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.571451 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.571463 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.571524 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.572486 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.573042 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.574373 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.574688 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.574757 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.576969 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.576970 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.576971 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.576979 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.578403 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.578838 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.578840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.578840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.578902 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.579197 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.579198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.579198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.579360 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.579447 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.579718 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.579801 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:40:22.580077 11239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.570586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.131:50909)
0807 00:40:22.570807 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.570852 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.571348 (+ 496us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.571589 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.571592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.572193 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.572397 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.572401 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.573130 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.573131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.574258 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.574261 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.574765 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.574766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.574898 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.575427 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.575435 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.576868 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.578508 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.578515 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.578517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.578755 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.578978 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.578978 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.578979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.579109 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.579437 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.579444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.579583 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.579693 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.579841 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":134,"threads_started":1}
I20260807 00:40:22.580773 11237 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40717
I20260807 00:40:22.580860 11237 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:22.581007 11237 heartbeater.cc:511] Master 127.1.116.190:40717 requested a full tablet report, sending...
I20260807 00:40:22.581451 10570 ts_manager.cc:194] Registered new tserver with Master: 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131:37721)
I20260807 00:40:22.581916 10570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:50909
I20260807 00:40:22.586033 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.116.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.189.keytab.
Entry for principal oryx/127.1.116.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.189.keytab.
I20260807 00:40:22.613348 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:34407
--webserver_interface=127.1.116.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.189.keytab
--principal=oryx/127.1.116.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:40717,127.1.116.189:34407
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:22.703311 11244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:22.703544 11244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:22.703572 11244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:22.704768 11244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:22.704813 11244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:22.704828 11244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:22.704841 11244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:22.704900 11244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:22.706425 11244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:40717,127.1.116.189:34407
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.189.keytab
--principal=oryx/127.1.116.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:34407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.116.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
W20260807 00:40:22.706605 11244 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.116.190:40717,127.1.116.189:34407'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 00:40:22.706776 11244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:22.707037 11244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:22.712292 11256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.712318 11253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.712318 11252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:22.712620 11244 server_base.cc:1034] running on GCE node
I20260807 00:40:22.712884 11244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:22.713214 11244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:22.714388 11244 hybrid_clock.cc:648] HybridClock initialized: now 1786063222714371 us; error 36 us; skew 500 ppm
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:22.717427 11244 init.cc:377] Logged in from keytab as oryx/127.1.116.189@KRBTEST.COM (short username oryx)
I20260807 00:40:22.717901 11244 webserver.cc:460] Webserver started at http://127.1.116.189:35883/ using document root <none> and password file <none>
I20260807 00:40:22.718138 11244 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:22.718207 11244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:22.718359 11244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:22.719575 11244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/instance:
uuid: "5fff16d053f545a79859832debbe9cf7"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "ae4e064998ba7a86e1c52ce5a9cb9c87"
server_key_iv: "3b2504b52b4c41a402bf1334c3608dcd"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.720001 11244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal/instance:
uuid: "5fff16d053f545a79859832debbe9cf7"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "ae4e064998ba7a86e1c52ce5a9cb9c87"
server_key_iv: "3b2504b52b4c41a402bf1334c3608dcd"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.721465 11244 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:22.722452 11263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.722925 11244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:22.723040 11244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
uuid: "5fff16d053f545a79859832debbe9cf7"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "ae4e064998ba7a86e1c52ce5a9cb9c87"
server_key_iv: "3b2504b52b4c41a402bf1334c3608dcd"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.723161 11244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:22.748984 11244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:22.751608 11244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:22.751784 11244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:22.757000 11244 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.189:34407
I20260807 00:40:22.757056 11338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.189:34407 every 8 connection(s)
I20260807 00:40:22.757548 11244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
I20260807 00:40:22.758729 11339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:22.759471 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 11244
I20260807 00:40:22.760874 11339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.189@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.116.189@KRBTEST.COM
I20260807 00:40:22.771785 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.761853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.1.116.190:40717)
0807 00:40:22.761913 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.761916 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.762087 (+ 171us) server_negotiation.cc:407] Connection header received
0807 00:40:22.762162 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.762163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.762175 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.762229 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.762925 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.763694 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.765002 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.765289 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.765360 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.768246 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.768247 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.768248 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.768256 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.770145 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.770665 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.770666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.770672 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.770743 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.771020 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.771021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.771021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.771207 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.771267 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.771568 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.771686 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:40:22.772073 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.761781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:60934)
0807 00:40:22.761985 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.762020 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.762106 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.762302 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.762305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.762673 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.762808 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.762815 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.763815 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.763816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.764915 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.764917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.765381 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.765382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.765458 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.766056 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.766068 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.768143 (+ 2075us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.770241 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.770247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.770249 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.770480 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.770823 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.770823 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.770823 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.770929 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.771276 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.771280 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.771389 (+ 109us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.771500 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.771681 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":23,"thread_start_us":106,"threads_started":1}
I20260807 00:40:22.772729 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.760225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37748 (local address 127.1.116.189:34407)
0807 00:40:22.760463 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.760474 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.760511 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:40:22.761078 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.761087 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.761153 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.761317 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.762247 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.763633 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.764805 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.765258 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.765380 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.769001 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.769010 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.769010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.769018 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.770927 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.771575 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.771578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.771579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.771653 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.771865 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.771866 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.771866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.772077 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.772247 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.772514 (+ 267us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.772620 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
I20260807 00:40:22.772949 11339 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.189@KRBTEST.COM for oryx/127.1.116.189@KRBTEST.COM
I20260807 00:40:22.779310 11344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.773220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.0.0.1:37758)
0807 00:40:22.773281 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.773294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.773341 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.773593 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.773594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.773703 (+ 109us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.773749 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.773750 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.774355 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.774355 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.774888 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.774890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.775243 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.775244 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.775305 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.775699 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.775704 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.776799 (+ 1095us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.778162 (+ 1363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.778163 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.778164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.778323 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.778597 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.778599 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.778599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.778681 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.779017 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.779018 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.779059 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.779143 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.779232 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 00:40:22.779428 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.773262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.1.116.189:34407)
0807 00:40:22.773316 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.773318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.773328 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:22.773506 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.773506 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.773517 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.773582 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.773821 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.774284 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.774962 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.775170 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.775228 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.776885 (+ 1657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.776886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.776886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.776892 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.778083 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.778393 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.778393 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.778394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.778441 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.778806 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.778807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.778807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.778921 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.778982 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.779164 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.779314 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":32}
I20260807 00:40:22.780748 11339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Bootstrap starting.
I20260807 00:40:22.781558 11339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:22.781791 11339 log.cc:824] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:22.782608 11339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: No bootstrap required, opened a new log
I20260807 00:40:22.783922 11339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } }
I20260807 00:40:22.784057 11339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:22.784082 11339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fff16d053f545a79859832debbe9cf7, State: Initialized, Role: FOLLOWER
I20260807 00:40:22.784204 11339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } }
I20260807 00:40:22.784466 11346 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } } }
I20260807 00:40:22.784524 11339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:22.784572 11346 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 00:40:22.786798 11357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:40:22.786885 11357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:22.788667 11244 master_runner.cc:428] Detected that this master 5fff16d053f545a79859832debbe9cf7 is joining an existing cluster
I20260807 00:40:22.788789 11244 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.116.189:34407
I20260807 00:40:22.797937 11369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.789922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:60942)
0807 00:40:22.790194 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.790208 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.790339 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.790501 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.790502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.790821 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.790945 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.790947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.791956 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.791956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.792652 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.792654 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.793092 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.793093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.793180 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.793711 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.793716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.794174 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.796533 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.796534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.796534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.796881 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.797098 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.797099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.797099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.797171 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.797509 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.797510 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.797577 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.797700 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.797820 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":199,"threads_started":1}
I20260807 00:40:22.797968 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.789999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60942 (local address 127.1.116.190:40717)
0807 00:40:22.790072 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.790075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.790281 (+ 206us) server_negotiation.cc:407] Connection header received
0807 00:40:22.790427 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.790428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.790441 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.790492 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.791060 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.791856 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.792754 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.792988 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.793066 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.794278 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.794279 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.794279 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.794286 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.796247 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.796958 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.796959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.796960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.797027 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.797243 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.797244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.797244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.797413 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.797495 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.797743 (+ 248us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.797840 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:40:22.798591 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.790187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37770 (local address 127.1.116.189:34407)
0807 00:40:22.790265 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.790267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.790419 (+ 152us) server_negotiation.cc:407] Connection header received
0807 00:40:22.790453 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.790454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.790465 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.790620 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.791985 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.792749 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.793855 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.794139 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.794224 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.795508 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.795509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.795510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.795516 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.797261 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.797665 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.797666 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.797666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.797724 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.797912 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.797912 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.797913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.798026 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.798075 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.798307 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.798427 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":94,"server-negotiator.queue_time_us":54}
I20260807 00:40:22.798604 11367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.789981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.0.0.1:37770)
0807 00:40:22.790265 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.790274 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.790338 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.790633 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.790634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.790875 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.790945 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.790947 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.792863 (+ 1916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.792864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.793753 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.793755 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.794245 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.794246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.794334 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.794946 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.794951 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.795378 (+ 427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.797351 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.797352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.797352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.797583 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.797795 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.797796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.797796 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.797850 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.798120 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.798121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.798173 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.798239 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.798358 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":2,"thread_start_us":95,"threads_started":1}
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063221, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.190@KRBTEST.COM for oryx/127.1.116.189@KRBTEST.COM
I20260807 00:40:22.809551 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.801232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.0.0.1:37782)
0807 00:40:22.801413 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.801456 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.801541 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.801719 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.801721 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.801982 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.802040 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.802041 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.802962 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.802963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.803610 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.803612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.804135 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.804137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.804218 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.804729 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.804734 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.806418 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.808250 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.808268 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.808270 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.808546 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.808817 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.808818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.808819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.808885 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.809215 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.809226 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.809276 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.809356 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.809462 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":124,"threads_started":1}
I20260807 00:40:22.809571 11343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.801292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37782 (local address 127.1.116.189:34407)
0807 00:40:22.801334 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.801335 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.801548 (+ 213us) server_negotiation.cc:407] Connection header received
0807 00:40:22.801630 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.801631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.801641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.801713 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.802121 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.802840 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.803704 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.804005 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.804080 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.806514 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.806515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.806515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.806523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.808146 (+ 1623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.808635 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.808636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.808636 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.808695 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.808974 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.808975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.808975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.809101 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.809177 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.809410 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.809486 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:40:22.810631 10570 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.116.189:34407: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5fff16d053f545a79859832debbe9cf7"
member_type: NON_VOTER
last_known_addr {
host: "127.1.116.189"
port: 34407
}
attrs {
promote: true
}
}
dest_uuid: "c01d75845cb043868f37d813b9b698dc"
cas_config_opid_index: -1
I20260807 00:40:22.810997 10570 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } }
I20260807 00:40:22.811738 11372 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 5fff16d053f545a79859832debbe9cf7 (127.1.116.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } }
I20260807 00:40:22.812103 11298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:22.813097 11372 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.116.189:34407
I20260807 00:40:22.813192 11372 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } }
I20260807 00:40:22.813270 11372 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:22.813303 11298 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Refusing update from remote peer c01d75845cb043868f37d813b9b698dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260807 00:40:22.813400 11244 master.cc:537] Master@127.1.116.189:34407 shutting down...
I20260807 00:40:22.813681 11371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 00:40:22.814592 10547 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 00:40:22.815060 10547 consensus_peers.cc:487] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc -> Peer 5fff16d053f545a79859832debbe9cf7 (127.1.116.189:34407): Couldn't send request to peer 5fff16d053f545a79859832debbe9cf7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 00:40:22.816880 11244 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:22.816988 11244 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:22.817055 11244 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: stopping tablet replica
I20260807 00:40:22.830736 11244 master.cc:552] Master@127.1.116.189:34407 shutdown complete.
I20260807 00:40:22.830807 11244 master_runner.cc:305] Clearing existing system tablet
I20260807 00:40:22.830945 11244 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:22.831012 11244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:22.831756 11244 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:22.832384 11377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.832533 11244 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:22.832607 11244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:40:22.832626 11244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
uuid: "5fff16d053f545a79859832debbe9cf7"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "ae4e064998ba7a86e1c52ce5a9cb9c87"
server_key_iv: "3b2504b52b4c41a402bf1334c3608dcd"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.833532 11244 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:22.834856 11244 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 00:40:22.834919 11244 log.cc:1196] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 00:40:22.835130 11244 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Deleting consensus metadata
I20260807 00:40:22.835302 11244 master_runner.cc:315] Copying system tablet from 127.1.116.190:40717
I20260807 00:40:22.836050 11244 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: tablet copy: Beginning tablet copy session from remote peer at address 127.1.116.190:40717
I20260807 00:40:22.842741 11387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.836399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:60956)
0807 00:40:22.836552 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:22.836566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:22.836680 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:22.836830 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:22.836831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:22.837047 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:22.837110 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.837112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.837832 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.837833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.838418 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:22.838420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.838864 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.838865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:22.838942 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.839493 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:22.839497 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:22.839807 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:22.841596 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.841597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.841598 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.841780 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.842007 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:22.842008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:22.842009 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:22.842072 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:22.842347 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:22.842349 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:22.842394 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 00:40:22.842482 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:40:22.842645 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260807 00:40:22.842736 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.836493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60956 (local address 127.1.116.190:40717)
0807 00:40:22.836554 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.836557 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.836664 (+ 107us) server_negotiation.cc:407] Connection header received
0807 00:40:22.836750 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.836751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.836763 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.836822 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.837197 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.837742 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.838511 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.838772 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.838863 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.839921 (+ 1058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.839922 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.839923 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.839930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.841496 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.841871 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.841872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.841873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.841938 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.842145 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.842146 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.842146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.842267 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.842344 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.842512 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.842623 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:40:22.843111 10598 tablet_copy_service.cc:140] P c01d75845cb043868f37d813b9b698dc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5fff16d053f545a79859832debbe9cf7 ({username='oryx', principal='oryx/127.1.116.189@KRBTEST.COM'} at 127.0.0.1:60956)
I20260807 00:40:22.843283 10598 tablet_copy_service.cc:161] P c01d75845cb043868f37d813b9b698dc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5fff16d053f545a79859832debbe9cf7 at {username='oryx', principal='oryx/127.1.116.189@KRBTEST.COM'} at 127.0.0.1:60956: session id = 5fff16d053f545a79859832debbe9cf7-00000000000000000000000000000000
I20260807 00:40:22.844096 10598 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 00:40:22.844852 11244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 00:40:22.845963 11244 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: tablet copy: Starting download of 0 data blocks...
I20260807 00:40:22.846136 11244 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: tablet copy: Starting download of 1 WAL segments...
I20260807 00:40:22.847222 11244 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 00:40:22.848110 10598 tablet_copy_service.cc:342] P c01d75845cb043868f37d813b9b698dc: Request end of tablet copy session 5fff16d053f545a79859832debbe9cf7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.116.189@KRBTEST.COM'} at 127.0.0.1:60956
I20260807 00:40:22.848181 10598 tablet_copy_service.cc:433] P c01d75845cb043868f37d813b9b698dc: ending tablet copy session 5fff16d053f545a79859832debbe9cf7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5fff16d053f545a79859832debbe9cf7
I20260807 00:40:22.851629 11244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:22.852943 11389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.852965 11388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:22.855664 11391 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:22.855769 11244 server_base.cc:1034] running on GCE node
I20260807 00:40:22.855877 11244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:22.856060 11244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:22.857204 11244 hybrid_clock.cc:648] HybridClock initialized: now 1786063222857187 us; error 34 us; skew 500 ppm
Aug 07 00:40:22 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063222, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:22.858929 11244 init.cc:377] Logged in from keytab as oryx/127.1.116.189@KRBTEST.COM (short username oryx)
I20260807 00:40:22.859270 11244 webserver.cc:460] Webserver started at http://127.1.116.189:46225/ using document root <none> and password file <none>
I20260807 00:40:22.859413 11244 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:22.859462 11244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:22.860318 11244 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:22.860987 11399 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:22.861212 11244 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:22.861279 11244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
uuid: "5fff16d053f545a79859832debbe9cf7"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "ae4e064998ba7a86e1c52ce5a9cb9c87"
server_key_iv: "3b2504b52b4c41a402bf1334c3608dcd"
server_key_version: "encryptionkey@0"
I20260807 00:40:22.861347 11244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:22.873600 11244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:22.875308 11244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:22.878831 11244 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.189:34407
I20260807 00:40:22.878937 11471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.189:34407 every 8 connection(s)
I20260807 00:40:22.879254 11244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
I20260807 00:40:22.880479 11473 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:40:22.880991 11473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Bootstrap starting.
I20260807 00:40:22.883790 11473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:40:22.884027 11473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Bootstrap complete.
I20260807 00:40:22.884269 11473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } }
I20260807 00:40:22.884380 11473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5fff16d053f545a79859832debbe9cf7, State: Initialized, Role: LEARNER
I20260807 00:40:22.884428 11473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } }
I20260807 00:40:22.884660 11478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } }
I20260807 00:40:22.884712 11473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:22.884763 11478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:40:22.886353 11497 catalog_manager.cc:1259] Loaded cluster ID: e4574c42febf434f812f8750e8f37439
I20260807 00:40:22.886452 11497 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: loading cluster ID for follower catalog manager: success
I20260807 00:40:22.887542 11497 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: acquiring CA information for follower catalog manager: success
I20260807 00:40:22.887868 11497 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:40:22.932792 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:22.925689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37804 (local address 127.1.116.189:34407)
0807 00:40:22.925858 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:22.925860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:22.925867 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:40:22.925897 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:22.925897 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:22.925908 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:22.925985 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:22.926731 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.927412 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.928183 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:22.928389 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:22.928470 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:22.929809 (+ 1339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:22.929811 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:22.929811 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:22.929819 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:22.931533 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.931938 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.931939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.931940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.932010 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:22.932218 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:22.932219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:22.932219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:22.932346 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:22.932469 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:22.932557 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 00:40:22.932662 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for oryx/127.1.116.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.188.keytab.
Entry for principal oryx/127.1.116.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.188.keytab.
I20260807 00:40:22.961506 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:43835
--webserver_interface=127.1.116.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.188.keytab
--principal=oryx/127.1.116.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:23.052743 11503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:23.052964 11503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:23.053001 11503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:23.054122 11503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:23.054185 11503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:23.054212 11503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:23.054244 11503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:23.054270 11503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:23.055861 11503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.188.keytab
--principal=oryx/127.1.116.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:43835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.116.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:23.056130 11503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:23.056350 11503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:23.061697 11509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.061695 11510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.061777 11514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:23.062153 11503 server_base.cc:1034] running on GCE node
I20260807 00:40:23.062390 11503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:23.062767 11503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:23.063930 11503 hybrid_clock.cc:648] HybridClock initialized: now 1786063223063924 us; error 47 us; skew 500 ppm
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:23.066628 11503 init.cc:377] Logged in from keytab as oryx/127.1.116.188@KRBTEST.COM (short username oryx)
I20260807 00:40:23.067086 11503 webserver.cc:460] Webserver started at http://127.1.116.188:41385/ using document root <none> and password file <none>
I20260807 00:40:23.067332 11503 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:23.067400 11503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:23.067559 11503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:23.068552 11503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/instance:
uuid: "76a774e511124cfc847280da621697c8"
format_stamp: "Formatted at 2026-08-07 00:40:23 on dist-test-slave-h6n0"
server_key: "206993f8917b1d0e9f5dc5ba092c7d9e"
server_key_iv: "f95901532bf8544eff952dc9893c53a5"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.068837 11503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal/instance:
uuid: "76a774e511124cfc847280da621697c8"
format_stamp: "Formatted at 2026-08-07 00:40:23 on dist-test-slave-h6n0"
server_key: "206993f8917b1d0e9f5dc5ba092c7d9e"
server_key_iv: "f95901532bf8544eff952dc9893c53a5"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.070142 11503 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:23.071002 11523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.071252 11503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:23.071332 11503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
uuid: "76a774e511124cfc847280da621697c8"
format_stamp: "Formatted at 2026-08-07 00:40:23 on dist-test-slave-h6n0"
server_key: "206993f8917b1d0e9f5dc5ba092c7d9e"
server_key_iv: "f95901532bf8544eff952dc9893c53a5"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.071429 11503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:23.087191 11503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:23.089262 11503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:23.089409 11503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:23.094738 11503 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.188:43835
I20260807 00:40:23.094813 11587 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.188:43835 every 8 connection(s)
I20260807 00:40:23.095170 11503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
I20260807 00:40:23.096433 11589 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:23.098222 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 11503
I20260807 00:40:23.098681 11589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.188@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.116.188@KRBTEST.COM
I20260807 00:40:23.110276 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.099629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60958 (local address 127.1.116.190:40717)
0807 00:40:23.099685 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.099688 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.099852 (+ 164us) server_negotiation.cc:407] Connection header received
0807 00:40:23.100055 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.100057 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.100069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.100135 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.101104 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.101923 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.103335 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.103638 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.103717 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.106740 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.106742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.106743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.106752 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.108561 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.109135 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.109137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.109137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.109230 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.109475 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.109476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.109477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.109638 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.109725 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.110084 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.110171 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 00:40:23.110616 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.099462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:60958)
0807 00:40:23.099680 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.099721 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.099966 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.100188 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.100191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.100675 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.100996 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.101002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.102037 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.102038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.103243 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.103246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.103759 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.103760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.103875 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.104456 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.104465 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.106581 (+ 2116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.108670 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.108677 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.108679 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.109042 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.109291 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.109292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.109292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.109394 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.109713 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.109716 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.109892 (+ 176us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.110018 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.110236 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
I20260807 00:40:23.110765 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.099017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48410 (local address 127.1.116.188:43835)
0807 00:40:23.099276 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.099283 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.099307 (+ 24us) server_negotiation.cc:407] Connection header received
0807 00:40:23.099967 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.099975 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.099999 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.100129 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.101018 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.102210 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.103335 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.103819 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.103879 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.107262 (+ 3383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.107270 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.107270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.107276 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.109138 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.109539 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.109541 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.109541 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.109583 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.109808 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.109809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.109810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.109969 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.110123 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.110263 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.110666 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":128,"threads_started":1}
I20260807 00:40:23.111416 11589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.188@KRBTEST.COM for oryx/127.1.116.189@KRBTEST.COM
I20260807 00:40:23.118170 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.111678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.0.0.1:37818)
0807 00:40:23.111750 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.111762 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.111807 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.112023 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.112024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.112145 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.112218 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.112219 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.112995 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.112996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.113570 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.113572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.114015 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.114016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.114069 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.114455 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.114459 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.115723 (+ 1264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.117271 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.117272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.117272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.117437 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.117653 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.117654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.117655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.117714 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.117944 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.117944 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.117979 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.118035 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.118108 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 00:40:23.118251 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.111823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37818 (local address 127.1.116.189:34407)
0807 00:40:23.111903 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.111905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.111914 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:23.111953 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.111954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.111968 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.112012 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.112274 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.112899 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.113673 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.113934 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.114004 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.115825 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.115826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.115826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.115832 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.117171 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.117514 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.117514 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.117515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.117568 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.117781 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.117781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.117782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.117879 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.117933 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.118114 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.118185 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:40:23.119050 11589 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.188@KRBTEST.COM for oryx/127.1.116.188@KRBTEST.COM
I20260807 00:40:23.125389 11594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.119349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.0.0.1:48412)
0807 00:40:23.119409 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.119423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.119476 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.119593 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.119594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.119700 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.119728 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.119728 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.120550 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.120551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.121015 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.121016 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.121390 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.121390 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.121428 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.121809 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.121813 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.122994 (+ 1181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.124427 (+ 1433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.124428 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.124428 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.124613 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.124813 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.124813 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.124813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.124864 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.125154 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.125154 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.125199 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.125253 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.125336 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 00:40:23.125469 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.119381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.1.116.188:43835)
0807 00:40:23.119434 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.119436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.119468 (+ 32us) server_negotiation.cc:407] Connection header received
0807 00:40:23.119520 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.119520 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.119528 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.119581 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.119795 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.120462 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.121088 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.121316 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.121372 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.123078 (+ 1706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.123079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.123079 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.123085 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.124331 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.124692 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.124693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.124693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.124743 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.124946 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.124947 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.124947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.125052 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.125151 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.125282 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.125388 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":33}
I20260807 00:40:23.126848 11589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Bootstrap starting.
I20260807 00:40:23.127727 11589 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:23.128109 11589 log.cc:824] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:23.128995 11589 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: No bootstrap required, opened a new log
I20260807 00:40:23.130398 11589 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } }
I20260807 00:40:23.130537 11589 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:23.130637 11589 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76a774e511124cfc847280da621697c8, State: Initialized, Role: FOLLOWER
I20260807 00:40:23.130771 11589 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } }
I20260807 00:40:23.131146 11589 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:23.131340 11596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } } }
I20260807 00:40:23.131445 11596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 00:40:23.133433 11608 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:40:23.133499 11608 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:23.136276 11503 master_runner.cc:428] Detected that this master 76a774e511124cfc847280da621697c8 is joining an existing cluster
I20260807 00:40:23.136330 11503 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.116.188:43835
I20260807 00:40:23.144987 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.137621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60970 (local address 127.1.116.190:40717)
0807 00:40:23.137669 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.137671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.137890 (+ 219us) server_negotiation.cc:407] Connection header received
0807 00:40:23.137993 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.137994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.138013 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.138089 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.138536 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.139392 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.140563 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.140866 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.140906 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.142089 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.142091 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.142091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.142098 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.143735 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.144124 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.144125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.144125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.144216 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.144437 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.144438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.144438 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.144556 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.144637 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.144809 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.144887 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 00:40:23.145018 11620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.137673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:60970)
0807 00:40:23.137821 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.137829 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.137912 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.138095 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.138096 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.138315 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.138434 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.138435 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.139487 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.139488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.140460 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.140463 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.141009 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.141010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.141084 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.141610 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.141616 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.141972 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.143857 (+ 1885us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.143858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.143858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.144045 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.144297 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.144298 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.144298 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.144366 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.144628 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.144629 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.144682 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.144766 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.144890 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063221, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.190@KRBTEST.COM for oryx/127.1.116.188@KRBTEST.COM
I20260807 00:40:23.162497 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 10663
I20260807 00:40:23.172465 11616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.137638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.0.0.1:37824)
0807 00:40:23.137807 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.137820 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.137891 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.138337 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.138338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.138582 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.138699 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.138701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.139650 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.139650 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.140520 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.140521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.141020 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.141021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.141112 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.165861 (+ 24749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.165868 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.166482 (+ 614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.171095 (+ 4613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.171096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.171097 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.171379 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.171602 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.171602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.171603 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.171668 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.171987 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.171988 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.172044 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.172110 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.172271 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":18,"thread_start_us":98,"threads_started":1}
I20260807 00:40:23.172868 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.138111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37824 (local address 127.1.116.189:34407)
0807 00:40:23.138193 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.138195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.138204 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:23.138231 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.138232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.138243 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.138317 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.138817 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.139560 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.140611 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.140906 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.140947 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.166734 (+ 25787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.166736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.166737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.166749 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.170985 (+ 4236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.171463 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.171464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.171464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.171524 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.171730 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.171731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.171731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.171901 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.171961 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.172620 (+ 659us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.172734 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:40:23.174914 11593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.138153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48422 (local address 127.1.116.188:43835)
0807 00:40:23.138369 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.138371 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.138509 (+ 138us) server_negotiation.cc:407] Connection header received
0807 00:40:23.138618 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.138619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.138626 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.138652 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.139054 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.139751 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.140846 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.141156 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.141194 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.171251 (+ 30057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.171253 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.171253 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.171260 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.172922 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.173962 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.173964 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.173964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.174019 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.174212 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.174213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.174214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.174346 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.174386 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.174680 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.174777 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":198}
I20260807 00:40:23.175299 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.137910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.0.0.1:48422)
0807 00:40:23.138457 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.138466 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.138551 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.138704 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.138705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.138896 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.138971 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.138973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.139841 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.139841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.140753 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.140756 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.141262 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.141263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.141317 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.169786 (+ 28469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.169792 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.170607 (+ 815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.173515 (+ 2908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.173516 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.173516 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.173876 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.174084 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.174085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.174085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.174154 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.174425 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.174425 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.174569 (+ 144us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.175000 (+ 431us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.175122 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":1,"thread_start_us":65,"threads_started":1}
I20260807 00:40:23.175927 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.147467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.0.0.1:48434)
0807 00:40:23.147521 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.147531 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.147571 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.147852 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.147853 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.148014 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.148055 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.148057 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.148964 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.148965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.149590 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.149592 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.149988 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.149989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.150062 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.150497 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.150502 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.151938 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.174082 (+ 22144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.174084 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.174084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.174359 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.174675 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.174676 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.174677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.174739 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.175584 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.175585 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.175640 (+ 55us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.175690 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.175814 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 00:40:23.176375 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.147531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.1.116.188:43835)
0807 00:40:23.147714 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.147716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.147722 (+ 6us) server_negotiation.cc:407] Connection header received
0807 00:40:23.147752 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.147753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.147761 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.147837 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.148166 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.148879 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.149672 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.149899 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.149985 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.152062 (+ 2077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.152064 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.152064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.152084 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.173963 (+ 21879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.174511 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.174512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.174512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.174596 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.174840 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.174841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.174841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.175294 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.176136 (+ 842us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.176180 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.176270 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
I20260807 00:40:23.176815 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:37929
--local_ip_for_outbound_sockets=127.1.116.129
--tserver_master_addrs=127.1.116.190:40717
--webserver_port=43629
--webserver_interface=127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab
--principal=oryx/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:40:23.177008 10570 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.116.188:43835: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "76a774e511124cfc847280da621697c8"
member_type: NON_VOTER
last_known_addr {
host: "127.1.116.188"
port: 43835
}
attrs {
promote: true
}
}
dest_uuid: "c01d75845cb043868f37d813b9b698dc"
cas_config_opid_index: 5
I20260807 00:40:23.177299 10570 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } }
I20260807 00:40:23.177950 11371 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 76a774e511124cfc847280da621697c8 (127.1.116.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.179243 11559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:23.179278 11371 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.116.188:43835
I20260807 00:40:23.179378 11371 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.179467 11371 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:23.179807 11503 master.cc:537] Master@127.1.116.188:43835 shutting down...
I20260807 00:40:23.180576 11559 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Refusing update from remote peer c01d75845cb043868f37d813b9b698dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 00:40:23.180974 11372 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 00:40:23.181663 10547 consensus_peers.cc:487] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc -> Peer 76a774e511124cfc847280da621697c8 (127.1.116.188:43835): Couldn't send request to peer 76a774e511124cfc847280da621697c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 00:40:23.186028 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.177818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.0.0.1:37826)
0807 00:40:23.181179 (+ 3361us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.181193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.181253 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.182098 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.182100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.182102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:23.182164 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.182166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.183366 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.183367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.184662 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.184666 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.185706 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.185708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.185809 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.185830 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.185898 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.185955 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3341}
I20260807 00:40:23.186100 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.177894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37826 (local address 127.1.116.189:34407)
0807 00:40:23.177965 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.177967 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.181471 (+ 3504us) server_negotiation.cc:407] Connection header received
0807 00:40:23.181498 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.181499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.181529 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.181590 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:23.182274 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.183262 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.184765 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.185400 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.185503 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.185572 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.185925 (+ 353us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.186000 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 00:40:23.186347 11435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 LEARNER]: Refusing update from remote peer c01d75845cb043868f37d813b9b698dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 00:40:23.186592 11372 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260807 00:40:23.187270 11435 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 76a774e511124cfc847280da621697c8 (127.1.116.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.188504 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.188711 11478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01d75845cb043868f37d813b9b698dc. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: NON_VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: true } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.188868 11478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:40:23.188723 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:40:23.188978 11503 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:23.189075 11503 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:23.189133 11503 tablet_replica.cc:330] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: stopping tablet replica
I20260807 00:40:23.189698 11372 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: attempting to promote NON_VOTER 5fff16d053f545a79859832debbe9cf7 to VOTER
I20260807 00:40:23.189875 11372 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } }
I20260807 00:40:23.190770 11435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 LEARNER]: Refusing update from remote peer c01d75845cb043868f37d813b9b698dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260807 00:40:23.190999 10547 consensus_peers.cc:487] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc -> Peer 76a774e511124cfc847280da621697c8 (127.1.116.188:43835): Couldn't send request to peer 76a774e511124cfc847280da621697c8. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 00:40:23.191022 11371 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 00:40:23.192605 11371 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5fff16d053f545a79859832debbe9cf7 (127.1.116.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.193946 11371 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.194042 11372 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.194159 11372 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:23.194042 11371 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:23.194489 11435 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 5fff16d053f545a79859832debbe9cf7 (127.1.116.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.195886 11633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.196009 11633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:40:23.203601 11503 master.cc:552] Master@127.1.116.188:43835 shutdown complete.
I20260807 00:40:23.203660 11503 master_runner.cc:305] Clearing existing system tablet
I20260807 00:40:23.203805 11503 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:23.203857 11503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:23.204706 11503 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:40:23.205313 11638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.205428 11503 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:23.205476 11503 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.205497 11503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
uuid: "76a774e511124cfc847280da621697c8"
format_stamp: "Formatted at 2026-08-07 00:40:23 on dist-test-slave-h6n0"
server_key: "206993f8917b1d0e9f5dc5ba092c7d9e"
server_key_iv: "f95901532bf8544eff952dc9893c53a5"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.206404 11503 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:23.208063 11503 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 00:40:23.208163 11503 log.cc:1196] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 00:40:23.208372 11503 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Deleting consensus metadata
I20260807 00:40:23.208505 11503 master_runner.cc:315] Copying system tablet from 127.1.116.190:40717
I20260807 00:40:23.209201 11503 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: tablet copy: Beginning tablet copy session from remote peer at address 127.1.116.190:40717
I20260807 00:40:23.216928 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.209591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60980 (local address 127.1.116.190:40717)
0807 00:40:23.209643 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.209645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.209710 (+ 65us) server_negotiation.cc:407] Connection header received
0807 00:40:23.209786 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.209787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.209799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.209887 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.210271 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.211142 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.212178 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.212504 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.212570 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.213918 (+ 1348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.213919 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.213920 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.213927 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.215732 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.216123 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.216124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.216124 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.216193 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.216387 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.216387 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.216388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.216511 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.216552 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.216752 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.216831 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 00:40:23.217289 11646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.209504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:60980)
0807 00:40:23.209643 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.209653 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.209722 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.209912 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.209913 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.210120 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.210176 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.210178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.211224 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.211225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.212063 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.212066 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.212663 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.212665 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.212736 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.213356 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.213362 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.213802 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.215828 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.215829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.215829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.216041 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.216266 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.216267 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.216267 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.216323 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.216596 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.216596 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.216644 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.217058 (+ 414us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.217196 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260807 00:40:23.217584 10598 tablet_copy_service.cc:140] P c01d75845cb043868f37d813b9b698dc: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 76a774e511124cfc847280da621697c8 ({username='oryx', principal='oryx/127.1.116.188@KRBTEST.COM'} at 127.0.0.1:60980)
I20260807 00:40:23.217645 10598 tablet_copy_service.cc:161] P c01d75845cb043868f37d813b9b698dc: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 76a774e511124cfc847280da621697c8 at {username='oryx', principal='oryx/127.1.116.188@KRBTEST.COM'} at 127.0.0.1:60980: session id = 76a774e511124cfc847280da621697c8-00000000000000000000000000000000
I20260807 00:40:23.218307 10598 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 00:40:23.218875 11503 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 00:40:23.220441 11503 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: tablet copy: Starting download of 0 data blocks...
I20260807 00:40:23.220605 11503 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: tablet copy: Starting download of 1 WAL segments...
I20260807 00:40:23.221529 11503 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 00:40:23.222487 10598 tablet_copy_service.cc:342] P c01d75845cb043868f37d813b9b698dc: Request end of tablet copy session 76a774e511124cfc847280da621697c8-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.116.188@KRBTEST.COM'} at 127.0.0.1:60980
I20260807 00:40:23.222594 10598 tablet_copy_service.cc:433] P c01d75845cb043868f37d813b9b698dc: ending tablet copy session 76a774e511124cfc847280da621697c8-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 76a774e511124cfc847280da621697c8
I20260807 00:40:23.226208 11503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:23.227456 11647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.227532 11648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:23.228164 11503 server_base.cc:1034] running on GCE node
W20260807 00:40:23.235404 11650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:23.235628 11503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:23.235803 11503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:23.236927 11503 hybrid_clock.cc:648] HybridClock initialized: now 1786063223236899 us; error 49 us; skew 500 ppm
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:23.238708 11503 init.cc:377] Logged in from keytab as oryx/127.1.116.188@KRBTEST.COM (short username oryx)
I20260807 00:40:23.239027 11503 webserver.cc:460] Webserver started at http://127.1.116.188:37015/ using document root <none> and password file <none>
I20260807 00:40:23.239166 11503 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:23.239213 11503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:23.240199 11503 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:23.240911 11657 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.241091 11503 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:23.241158 11503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
uuid: "76a774e511124cfc847280da621697c8"
format_stamp: "Formatted at 2026-08-07 00:40:23 on dist-test-slave-h6n0"
server_key: "206993f8917b1d0e9f5dc5ba092c7d9e"
server_key_iv: "f95901532bf8544eff952dc9893c53a5"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.241227 11503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:23.252633 11503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:23.254866 11503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:23.258363 11503 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.188:43835
I20260807 00:40:23.258473 11729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.188:43835 every 8 connection(s)
I20260807 00:40:23.258752 11503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
I20260807 00:40:23.259878 11733 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:40:23.260406 11733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Bootstrap starting.
I20260807 00:40:23.263823 11733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:40:23.264189 11733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: Bootstrap complete.
I20260807 00:40:23.264552 11733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } }
I20260807 00:40:23.264755 11733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 76a774e511124cfc847280da621697c8, State: Initialized, Role: LEARNER
I20260807 00:40:23.264839 11733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } }
I20260807 00:40:23.265152 11733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:23.265173 11737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.265314 11737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:40:23.267251 11753 catalog_manager.cc:1259] Loaded cluster ID: e4574c42febf434f812f8750e8f37439
I20260807 00:40:23.267305 11753 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: loading cluster ID for follower catalog manager: success
I20260807 00:40:23.269066 11753 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: acquiring CA information for follower catalog manager: success
I20260807 00:40:23.269378 11753 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 00:40:23.292986 11628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:23.293246 11628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:23.293283 11628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:23.294463 11628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:23.294526 11628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:23.294596 11628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:23.296177 11628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab
--principal=oryx/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:37929
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=43629
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:23.296460 11628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:23.296716 11628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:23.302619 11764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.302649 11762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:23.302950 11628 server_base.cc:1034] running on GCE node
W20260807 00:40:23.303126 11761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:23.303390 11628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:23.303670 11628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:23.304852 11628 hybrid_clock.cc:648] HybridClock initialized: now 1786063223304844 us; error 43 us; skew 500 ppm
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:23.307791 11628 init.cc:377] Logged in from keytab as oryx/127.1.116.129@KRBTEST.COM (short username oryx)
I20260807 00:40:23.308379 11628 webserver.cc:460] Webserver started at http://127.1.116.129:43629/ using document root <none> and password file <none>
I20260807 00:40:23.308655 11628 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:23.308748 11628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:23.310384 11628 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:23.311230 11773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.311533 11628 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:23.311627 11628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "f3b1ba619ef64d509ea97bf5661c9799"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "f2d6a47ecb5d80f91bac2693b21716bd"
server_key_iv: "027878602ee0d73715a89a15fd014696"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.312232 11628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:23.321116 11628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:23.323486 11628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:23.323607 11628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:23.323799 11628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:23.324221 11628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:23.324270 11628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.324293 11628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:23.324347 11628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.332909 11628 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:37929
I20260807 00:40:23.333120 11928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:37929 every 8 connection(s)
I20260807 00:40:23.333408 11628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:23.337822 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 11628
I20260807 00:40:23.338017 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 10862
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.188@KRBTEST.COM
I20260807 00:40:23.345757 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.334360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:36979 (local address 127.1.116.188:43835)
0807 00:40:23.334568 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.334619 (+ 51us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.334817 (+ 198us) server_negotiation.cc:407] Connection header received
0807 00:40:23.335709 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.335711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.335725 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.335821 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.336631 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.337690 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.338916 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.339397 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.339913 (+ 516us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.342588 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.342590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.342590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.342598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.344049 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.344495 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.344496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.344497 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.344564 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.344825 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.344826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.344826 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.344956 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.345030 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.345435 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.345585 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":80,"threads_started":1}
I20260807 00:40:23.345890 11935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.334312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.1.116.129:36979)
0807 00:40:23.334748 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.334760 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.335414 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.335864 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.335864 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.336303 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.336507 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.336515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.337781 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.337783 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.338792 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.338797 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.339507 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.339509 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.339602 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.340434 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.340441 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.342468 (+ 2027us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.344151 (+ 1683us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.344158 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.344160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.344399 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.344632 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.344633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.344633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.344749 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.345034 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.345041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.345226 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.345340 (+ 114us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.345540 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":254,"thread_start_us":97,"threads_started":1}
I20260807 00:40:23.346374 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.334312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.129:60195)
0807 00:40:23.334596 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.334650 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.335414 (+ 764us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.335729 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.335733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.336413 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.336508 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.336515 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.337537 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.337538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.338774 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.338776 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.339172 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.339173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.339316 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.339916 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.339926 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.341962 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.345207 (+ 3245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.345208 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.345209 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.345416 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.345657 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.345657 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.345658 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.345712 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.346049 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.346050 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.346095 (+ 45us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.346173 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.346272 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":136,"threads_started":1}
I20260807 00:40:23.346374 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.334431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:60195 (local address 127.1.116.190:40717)
0807 00:40:23.334485 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.334488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.334756 (+ 268us) server_negotiation.cc:407] Connection header received
0807 00:40:23.335552 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.335554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.335572 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.335639 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.336631 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.337438 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.338868 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.339086 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.339164 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.342083 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.342085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.342086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.342094 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.345027 (+ 2933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.345514 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.345516 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.345516 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.345580 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.345779 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.345780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.345780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.345943 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.345997 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.346205 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.346285 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 00:40:23.354477 11930 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40717
I20260807 00:40:23.354614 11930 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.354833 11930 heartbeater.cc:511] Master 127.1.116.190:40717 requested a full tablet report, sending...
I20260807 00:40:23.355105 11929 heartbeater.cc:347] Connected to a master server at 127.1.116.188:43835
I20260807 00:40:23.355144 11929 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.355211 11929 heartbeater.cc:511] Master 127.1.116.188:43835 requested a full tablet report, sending...
I20260807 00:40:23.355571 10570 ts_manager.cc:194] Re-registered known tserver with Master: f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129:37929)
I20260807 00:40:23.356155 10570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:60195
I20260807 00:40:23.356222 11681 ts_manager.cc:194] Registered new tserver with Master: f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129:37929)
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.189@KRBTEST.COM
I20260807 00:40:23.361182 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:37965
--local_ip_for_outbound_sockets=127.1.116.130
--tserver_master_addrs=127.1.116.190:40717
--webserver_port=45279
--webserver_interface=127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab
--principal=oryx/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:40:23.361584 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.335118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:42753 (local address 127.1.116.189:34407)
0807 00:40:23.335169 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.335172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.335182 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:23.335562 (+ 380us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.335563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.335575 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.335617 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.336609 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.337149 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.356455 (+ 19306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.356756 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.356803 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.358923 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.358924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.358924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.358932 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.360354 (+ 1422us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.360763 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.360764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.360764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.360823 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.361025 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.361026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.361026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.361147 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.361228 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.361432 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.361515 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:40:23.362071 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.334845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.1.116.129:42753)
0807 00:40:23.335057 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.335070 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.335436 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.335789 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.335791 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.336261 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.336506 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.336513 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.337264 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.337265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.356305 (+ 19040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.356309 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.356888 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.356889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.356943 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.357414 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.357419 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.358827 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.360458 (+ 1631us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.360459 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.360460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.360674 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.360894 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.360894 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.360895 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.360950 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.361257 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.361258 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.361306 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.361695 (+ 389us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.361824 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":136,"threads_started":1}
I20260807 00:40:23.362648 11931 heartbeater.cc:347] Connected to a master server at 127.1.116.189:34407
I20260807 00:40:23.362704 11931 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.362794 11931 heartbeater.cc:511] Master 127.1.116.189:34407 requested a full tablet report, sending...
I20260807 00:40:23.363340 11420 ts_manager.cc:194] Registered new tserver with Master: f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129:37929)
W20260807 00:40:23.455216 11937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:23.455490 11937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:23.455533 11937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:23.456696 11937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:23.456738 11937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:23.456764 11937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:23.458349 11937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab
--principal=oryx/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:37965
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=45279
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:23.458655 11937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:23.458935 11937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:23.464574 11946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.464596 11949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.464566 11947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:23.465327 11937 server_base.cc:1034] running on GCE node
I20260807 00:40:23.465549 11937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:23.465821 11937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:23.467027 11937 hybrid_clock.cc:648] HybridClock initialized: now 1786063223467001 us; error 41 us; skew 500 ppm
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:23.469799 11937 init.cc:377] Logged in from keytab as oryx/127.1.116.130@KRBTEST.COM (short username oryx)
I20260807 00:40:23.470312 11937 webserver.cc:460] Webserver started at http://127.1.116.130:45279/ using document root <none> and password file <none>
I20260807 00:40:23.470646 11937 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:23.470741 11937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:23.472448 11937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:23.473250 11960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.473476 11937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:23.473587 11937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "e263fdb0d6a848f986ebcce331fc9c45"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "73c7c6fa6b1cf8d8c00f67ad124b88cc"
server_key_iv: "e88b12953882878fc8593aa4b2f3307d"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.474023 11937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:23.497553 11937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:23.499971 11937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:23.500159 11937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:23.500463 11937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:23.500830 11937 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:23.500888 11937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.500933 11937 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:23.500967 11937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.509075 11937 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:37965
I20260807 00:40:23.509130 12121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:37965 every 8 connection(s)
I20260807 00:40:23.509572 11937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.188@KRBTEST.COM
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.189@KRBTEST.COM
I20260807 00:40:23.519214 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 11937
I20260807 00:40:23.519402 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 11050
I20260807 00:40:23.523046 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.510401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:53153 (local address 127.1.116.190:40717)
0807 00:40:23.510450 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.510452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.510864 (+ 412us) server_negotiation.cc:407] Connection header received
0807 00:40:23.511648 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.511649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.511666 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.511720 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.513692 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.514230 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515174 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.515443 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515535 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.519081 (+ 3546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.519083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.519083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.519091 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.521151 (+ 2060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.521591 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.521593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.521593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.521651 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.522177 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.522179 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.522179 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.522325 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.522379 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.522824 (+ 445us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.522907 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:40:23.523811 12127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.510419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.130:53153)
0807 00:40:23.510756 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.510767 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.511524 (+ 757us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.511840 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.511844 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.512517 (+ 673us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.512795 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.512801 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.514322 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.514323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.515085 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.515087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.515546 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.515693 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.516301 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.516312 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.518948 (+ 2636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.521237 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.521245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.521247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.521520 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.521726 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.521726 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.521727 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.521878 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.522415 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.522423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.522635 (+ 212us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.523268 (+ 633us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.523509 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":12,"spinlock_wait_cycles":768,"thread_start_us":147,"threads_started":1}
I20260807 00:40:23.524475 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.510407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:48607 (local address 127.1.116.189:34407)
0807 00:40:23.510471 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.510474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.510833 (+ 359us) server_negotiation.cc:407] Connection header received
0807 00:40:23.511664 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.511665 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.511679 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.511713 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.512900 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.513513 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515196 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.515495 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515536 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.519362 (+ 3826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.519363 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.519364 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.519373 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.523155 (+ 3782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.523619 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.523621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.523621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.523675 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.523874 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.523875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.523875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.524021 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.524069 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.524283 (+ 214us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.524372 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:40:23.524842 12128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.510418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.1.116.130:48607)
0807 00:40:23.510675 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.510719 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.511532 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.511842 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.511845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.512444 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.512797 (+ 353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.512805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.513618 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.513619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.515092 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.515095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.515628 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.515704 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.516649 (+ 945us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.516653 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.519226 (+ 2573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.523270 (+ 4044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.523271 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.523271 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.523532 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.523741 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.523741 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.523742 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.523797 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.524101 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.524102 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.524144 (+ 42us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.524592 (+ 448us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.524722 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":81,"thread_start_us":161,"threads_started":1}
I20260807 00:40:23.525059 12123 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40717
I20260807 00:40:23.525197 12123 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.525427 12123 heartbeater.cc:511] Master 127.1.116.190:40717 requested a full tablet report, sending...
I20260807 00:40:23.525490 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.511000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:40791 (local address 127.1.116.188:43835)
0807 00:40:23.511084 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.511087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.511151 (+ 64us) server_negotiation.cc:407] Connection header received
0807 00:40:23.511662 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.511664 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.511682 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.511713 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.512906 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.513599 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515197 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.515553 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515595 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.518677 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.518679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.518679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.518686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.523420 (+ 4734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.524314 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.524315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.524316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.524406 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.524679 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.524681 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.524681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.524860 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.524916 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.525222 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.525332 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:40:23.525869 10570 ts_manager.cc:194] Re-registered known tserver with Master: e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965)
I20260807 00:40:23.526001 12129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.510906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.1.116.130:40791)
0807 00:40:23.511065 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.511076 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.511545 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.511842 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.511845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.512450 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.512797 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.512805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.513687 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.513687 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.515092 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.515095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.515667 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.515667 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.515777 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.516632 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.516638 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.518485 (+ 1847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.523851 (+ 5366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.523853 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.523854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.524194 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.524503 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.524503 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.524504 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.524582 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.524975 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.524976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.525052 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.525642 (+ 590us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.525802 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":103,"thread_start_us":84,"threads_started":1}
I20260807 00:40:23.526417 10570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:53153
I20260807 00:40:23.535840 12122 heartbeater.cc:347] Connected to a master server at 127.1.116.188:43835
I20260807 00:40:23.535917 12122 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.535943 12125 heartbeater.cc:347] Connected to a master server at 127.1.116.189:34407
I20260807 00:40:23.536007 12125 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.536013 12122 heartbeater.cc:511] Master 127.1.116.188:43835 requested a full tablet report, sending...
I20260807 00:40:23.536101 12125 heartbeater.cc:511] Master 127.1.116.189:34407 requested a full tablet report, sending...
I20260807 00:40:23.536402 11681 ts_manager.cc:194] Registered new tserver with Master: e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965)
I20260807 00:40:23.536451 11420 ts_manager.cc:194] Registered new tserver with Master: e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965)
I20260807 00:40:23.541538 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:37721
--local_ip_for_outbound_sockets=127.1.116.131
--tserver_master_addrs=127.1.116.190:40717
--webserver_port=38367
--webserver_interface=127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab
--principal=oryx/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:43315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:23.627461 12130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:23.627665 12130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:23.627688 12130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:23.628762 12130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:23.628803 12130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:23.628829 12130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:23.630199 12130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:43315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab
--principal=oryx/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:37721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=38367
--tserver_master_addrs=127.1.116.190:40717,127.1.116.189:34407,127.1.116.188:43835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:23.630368 12130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:23.630586 12130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:23.635890 12138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.635957 12136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:23.635921 12135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:23.635939 12130 server_base.cc:1034] running on GCE node
I20260807 00:40:23.636379 12130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:23.636624 12130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:23.637787 12130 hybrid_clock.cc:648] HybridClock initialized: now 1786063223637767 us; error 50 us; skew 500 ppm
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:23.640599 12130 init.cc:377] Logged in from keytab as oryx/127.1.116.131@KRBTEST.COM (short username oryx)
I20260807 00:40:23.641238 12130 webserver.cc:460] Webserver started at http://127.1.116.131:38367/ using document root <none> and password file <none>
I20260807 00:40:23.641490 12130 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:23.641577 12130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:23.643658 12130 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:23.644374 12147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.644593 12130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:23.644719 12130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "90eb9bf8a88d4e22974c0a9513b6241a"
format_stamp: "Formatted at 2026-08-07 00:40:22 on dist-test-slave-h6n0"
server_key: "7ec5c8edb5da1efda9588e521d3dd9d0"
server_key_iv: "1893234c2ad21d6da4c77ac8d387240b"
server_key_version: "encryptionkey@0"
I20260807 00:40:23.645061 12130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:23.655009 12130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:23.656814 12130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:23.656973 12130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:23.657186 12130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:23.657543 12130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:23.657608 12130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.657640 12130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:23.657667 12130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:23.665233 12130 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:37721
I20260807 00:40:23.665344 12298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:37721 every 8 connection(s)
I20260807 00:40:23.665693 12130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:23.667723 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 12130
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.189@KRBTEST.COM
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.188@KRBTEST.COM
I20260807 00:40:23.677583 12316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.669019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37842 (local address 127.1.116.189:34407)
0807 00:40:23.669146 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.669149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.669340 (+ 191us) server_negotiation.cc:407] Connection header received
0807 00:40:23.669368 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.669369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.669383 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.669467 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.670652 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.671318 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.672382 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.672678 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.672717 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.674476 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.674478 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.674478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.674485 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.676266 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.676673 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.676674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.676675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.676735 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.676931 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.676932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.676932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.677067 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.677147 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.677368 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.677446 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
Aug 07 00:40:23 dist-test-slave-h6n0 krb5kdc[10507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063223, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:23.680688 12306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.666870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.1.116.131:43537)
0807 00:40:23.667332 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.667344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.668555 (+ 1211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.668959 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.668964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.669523 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.669895 (+ 372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.669903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.671077 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.671078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.672358 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.672361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.672852 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.672853 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.673003 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.673983 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.673993 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.676199 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.678700 (+ 2501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.678707 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.678709 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.679070 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.679295 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.679295 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.679296 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.679416 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.679700 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.679706 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.679868 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.679974 (+ 106us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.680338 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":1,"thread_start_us":193,"threads_started":1}
I20260807 00:40:23.681183 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.667018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:43537 (local address 127.1.116.189:34407)
0807 00:40:23.667080 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.667083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.667426 (+ 343us) server_negotiation.cc:407] Connection header received
0807 00:40:23.668760 (+ 1334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.668761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.668775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.668830 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.670028 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.670977 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.672453 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.672740 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.672795 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.676323 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.676324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.676325 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.676331 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.678604 (+ 2273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.679161 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.679162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.679163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.679221 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.679492 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.679493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.679493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.679613 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.679672 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.680995 (+ 1323us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.681089 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 00:40:23.681195 12310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.666473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.1.116.131:59967)
0807 00:40:23.667203 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.667214 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.668576 (+ 1362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.669405 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.669408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.669714 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.669893 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.669905 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.670838 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.670839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.672363 (+ 1524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.672366 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.674510 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.674512 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.674657 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.675648 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.675654 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.677481 (+ 1827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.679814 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.679815 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.679815 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.680024 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.680417 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.680418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.680418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.680487 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.680810 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.680811 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.680858 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.680929 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.681085 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":44,"thread_start_us":175,"threads_started":1}
I20260807 00:40:23.681559 11933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.666509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:59967 (local address 127.1.116.188:43835)
0807 00:40:23.666639 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.666642 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.667281 (+ 639us) server_negotiation.cc:407] Connection header received
0807 00:40:23.669266 (+ 1985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.669267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.669278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.669328 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.669995 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.670704 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.673720 (+ 3016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.674062 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.674437 (+ 375us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.677583 (+ 3146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.677584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.677585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.677594 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.679737 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.680238 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.680240 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.680240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.680333 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.680554 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.680554 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.680555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.680704 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.680777 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.681337 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.681442 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260807 00:40:23.681653 12301 heartbeater.cc:347] Connected to a master server at 127.1.116.189:34407
I20260807 00:40:23.681749 12301 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.681918 12301 heartbeater.cc:511] Master 127.1.116.189:34407 requested a full tablet report, sending...
I20260807 00:40:23.682246 12299 heartbeater.cc:347] Connected to a master server at 127.1.116.188:43835
I20260807 00:40:23.682304 12299 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.682386 12299 heartbeater.cc:511] Master 127.1.116.188:43835 requested a full tablet report, sending...
I20260807 00:40:23.682382 11420 ts_manager.cc:194] Registered new tserver with Master: 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131:37721)
I20260807 00:40:23.682819 11681 ts_manager.cc:194] Registered new tserver with Master: 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131:37721)
I20260807 00:40:23.684752 12305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.666470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.131:51639)
0807 00:40:23.666745 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.666824 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.668549 (+ 1725us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.668959 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.668964 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.671244 (+ 2280us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:23.671332 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.671334 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.672427 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.672428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.673317 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.673320 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.673847 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.673849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.673933 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.674461 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:23.674468 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:23.678898 (+ 4430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:23.682931 (+ 4033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.682933 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.682933 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.683486 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.683721 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:23.683722 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:23.683722 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:23.683785 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:23.684129 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:23.684130 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:23.684170 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.684226 (+ 56us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.684343 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":585,"threads_started":1}
I20260807 00:40:23.685385 10628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.666608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51639 (local address 127.1.116.190:40717)
0807 00:40:23.666668 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.666671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.666944 (+ 273us) server_negotiation.cc:407] Connection header received
0807 00:40:23.668710 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.668711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.668728 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.668765 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.671446 (+ 2681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.672302 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.673422 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.673756 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.673851 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.679005 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.679007 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.679007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.679016 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.682796 (+ 3780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.683581 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.683582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.683583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.683647 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.683856 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.683857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.683857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.684010 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.684082 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.685202 (+ 1120us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.685286 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:40:23.685725 12300 heartbeater.cc:347] Connected to a master server at 127.1.116.190:40717
I20260807 00:40:23.685771 12300 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:23.685848 12300 heartbeater.cc:511] Master 127.1.116.190:40717 requested a full tablet report, sending...
I20260807 00:40:23.686094 10570 ts_manager.cc:194] Re-registered known tserver with Master: 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131:37721)
I20260807 00:40:23.686353 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.669616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60992 (local address 127.1.116.190:40717)
0807 00:40:23.669822 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.669825 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.673614 (+ 3789us) server_negotiation.cc:407] Connection header received
0807 00:40:23.673646 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.673647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.673664 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.673757 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.677262 (+ 3505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.678972 (+ 1710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.680296 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.680652 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.680727 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.682741 (+ 2014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.682742 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.682743 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.682750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.684644 (+ 1894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.685065 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.685066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.685066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.685132 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.685316 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.685317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.685318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.685430 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.685520 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.685992 (+ 472us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.686095 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
I20260807 00:40:23.686894 12318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.672958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48458 (local address 127.1.116.188:43835)
0807 00:40:23.681109 (+ 8151us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.681111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.681120 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:23.681142 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.681142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.681168 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.683128 (+ 1960us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:23.683323 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.684204 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.685700 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.686355 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.686657 (+ 302us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.686738 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.686759 (+ 21us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.686807 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8129,"thread_start_us":701,"threads_started":1}
I20260807 00:40:23.687213 10570 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:51639
I20260807 00:40:23.687445 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.672738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.0.0.1:48458)
0807 00:40:23.672834 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.672845 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.672884 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.683168 (+ 10284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.683169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.683172 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:23.683224 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.683225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.684309 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.684310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.685542 (+ 1232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.685545 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.686453 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.686454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.686519 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.686549 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.686581 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.687373 (+ 792us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":715}
I20260807 00:40:23.687820 11687 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 00:40:23.688062 11737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c01d75845cb043868f37d813b9b698dc. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.688239 11737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:40:23.688771 10570 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60992:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:23.691090 10570 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:23.691690 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.669259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.1.116.188:43835)
0807 00:40:23.683184 (+ 13925us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.683186 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.683194 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:40:23.683212 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.683213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.683223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.683298 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.683964 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.684767 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.686229 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.686513 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.686612 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.688268 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.688270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.688270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.688278 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.690169 (+ 1891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.690636 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.690637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.690638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.690711 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.691082 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.691083 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.691084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.691236 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.691338 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.691497 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.691600 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8352,"thread_start_us":88,"threads_started":1}
I20260807 00:40:23.692238 11635 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: NON_VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: true } } }
I20260807 00:40:23.692345 11635 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:23.694103 11635 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: attempting to promote NON_VOTER 76a774e511124cfc847280da621697c8 to VOTER
I20260807 00:40:23.694422 11635 consensus_queue.cc:237] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } }
I20260807 00:40:23.695315 11688 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 LEARNER]: Refusing update from remote peer c01d75845cb043868f37d813b9b698dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 00:40:23.695531 11372 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260807 00:40:23.695925 11435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Refusing update from remote peer c01d75845cb043868f37d813b9b698dc: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 00:40:23.696146 11372 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 00:40:23.696383 11372 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 76a774e511124cfc847280da621697c8 (127.1.116.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.696605 11687 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 76a774e511124cfc847280da621697c8 (127.1.116.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.697134 11435 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 76a774e511124cfc847280da621697c8 (127.1.116.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.697471 11372 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.697511 12328 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.697489 11371 sys_catalog.cc:455] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.697624 11371 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:23.697566 12328 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:23.697557 11372 sys_catalog.cc:458] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:23.698009 11737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.698107 11737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:40:23.698390 12338 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c01d75845cb043868f37d813b9b698dc" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:23.698509 12338 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:40:23.702648 11370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.698337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:37721 (local address 127.0.0.1:49984)
0807 00:40:23.698458 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.698470 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.698514 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.699243 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.699244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.699246 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:23.699291 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.699294 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.700467 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.700468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.701434 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.701436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.702372 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.702373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.702437 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.702441 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.702516 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.702587 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260807 00:40:23.702838 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.698652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49984 (local address 127.1.116.131:37721)
0807 00:40:23.699021 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.699026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.699040 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:40:23.699092 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.699095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.699119 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.699231 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:23.699380 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.700372 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.701535 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.702289 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.702354 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.702421 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.702572 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.702617 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":98,"threads_started":1}
I20260807 00:40:23.703445 12344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.698720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37929 (local address 127.0.0.1:44910)
0807 00:40:23.699499 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.699511 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.699617 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.699925 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.699926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.699929 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:23.699978 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.699981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.701188 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.701189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.702155 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.702157 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.703140 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.703141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.703231 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.703238 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.703307 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.703368 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":1,"thread_start_us":98,"threads_started":1}
I20260807 00:40:23.703456 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.698845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44910 (local address 127.1.116.129:37929)
0807 00:40:23.699050 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.699055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.699584 (+ 529us) server_negotiation.cc:407] Connection header received
0807 00:40:23.699745 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.699748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.699765 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.699824 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:23.700076 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.701099 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.702251 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.703049 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.703129 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.703205 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.703339 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.703377 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
I20260807 00:40:23.703765 12211 tablet_service.cc:1467] Processing CreateTablet for tablet 048362bcb5f24983b8054f64b69e82bd (DEFAULT_TABLE table=test-table [id=f1eff3e18da64b4281476effc5f3bf1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:23.704248 12211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 048362bcb5f24983b8054f64b69e82bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:23.704324 12342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.698406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:37965 (local address 127.0.0.1:55184)
0807 00:40:23.698900 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.698913 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.698975 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.699629 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.699629 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.699632 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:23.699673 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.699674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.701000 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.701001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.702138 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.702141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.704059 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.704060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.704138 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.704142 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.704200 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.704253 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":29,"thread_start_us":98,"threads_started":1}
I20260807 00:40:23.704244 11839 tablet_service.cc:1467] Processing CreateTablet for tablet 048362bcb5f24983b8054f64b69e82bd (DEFAULT_TABLE table=test-table [id=f1eff3e18da64b4281476effc5f3bf1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:23.704581 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 048362bcb5f24983b8054f64b69e82bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:23.704548 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.699026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55184 (local address 127.1.116.130:37965)
0807 00:40:23.699271 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.699277 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.699287 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:23.699328 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.699330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.699350 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.699509 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:23.699829 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.700873 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.703249 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.703953 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.704039 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.704112 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.704212 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.704250 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":127,"threads_started":1}
I20260807 00:40:23.705263 12028 tablet_service.cc:1467] Processing CreateTablet for tablet 048362bcb5f24983b8054f64b69e82bd (DEFAULT_TABLE table=test-table [id=f1eff3e18da64b4281476effc5f3bf1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:23.705544 12028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 048362bcb5f24983b8054f64b69e82bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:23.708689 12348 tablet_bootstrap.cc:492] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Bootstrap starting.
I20260807 00:40:23.709049 12350 tablet_bootstrap.cc:492] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Bootstrap starting.
I20260807 00:40:23.709054 12351 tablet_bootstrap.cc:492] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Bootstrap starting.
I20260807 00:40:23.709709 12350 tablet_bootstrap.cc:654] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:23.709918 12351 tablet_bootstrap.cc:654] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:23.710057 12350 log.cc:824] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:23.710305 12351 log.cc:824] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:23.710973 12350 tablet_bootstrap.cc:492] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: No bootstrap required, opened a new log
I20260807 00:40:23.711056 12350 ts_tablet_manager.cc:1397] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:23.711184 12351 tablet_bootstrap.cc:492] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: No bootstrap required, opened a new log
I20260807 00:40:23.711189 12348 tablet_bootstrap.cc:654] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:23.711259 12351 ts_tablet_manager.cc:1397] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 00:40:23.711544 12348 log.cc:824] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:23.712707 12348 tablet_bootstrap.cc:492] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: No bootstrap required, opened a new log
I20260807 00:40:23.712827 12348 ts_tablet_manager.cc:1397] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260807 00:40:23.713502 12350 raft_consensus.cc:348] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.713580 12351 raft_consensus.cc:348] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.713704 12350 raft_consensus.cc:374] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:23.713760 12351 raft_consensus.cc:374] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:23.713773 12350 raft_consensus.cc:729] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90eb9bf8a88d4e22974c0a9513b6241a, State: Initialized, Role: FOLLOWER
I20260807 00:40:23.713810 12351 raft_consensus.cc:729] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e263fdb0d6a848f986ebcce331fc9c45, State: Initialized, Role: FOLLOWER
I20260807 00:40:23.713913 12350 consensus_queue.cc:260] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.713958 12351 consensus_queue.cc:260] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.714244 12351 ts_tablet_manager.cc:1428] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:40:23.714502 12123 heartbeater.cc:503] Master 127.1.116.190:40717 was elected leader, sending a full tablet report...
I20260807 00:40:23.714483 12300 heartbeater.cc:503] Master 127.1.116.190:40717 was elected leader, sending a full tablet report...
I20260807 00:40:23.714208 12350 ts_tablet_manager.cc:1428] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:40:23.716775 12348 raft_consensus.cc:348] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.716903 12348 raft_consensus.cc:374] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:23.716924 12348 raft_consensus.cc:729] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3b1ba619ef64d509ea97bf5661c9799, State: Initialized, Role: FOLLOWER
I20260807 00:40:23.717038 12348 consensus_queue.cc:260] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.717312 12348 ts_tablet_manager.cc:1428] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260807 00:40:23.717336 11930 heartbeater.cc:503] Master 127.1.116.190:40717 was elected leader, sending a full tablet report...
W20260807 00:40:23.760782 12126 tablet.cc:2369] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:40:23.834487 11932 tablet.cc:2369] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:23.862716 12361 raft_consensus.cc:482] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:23.862828 12361 raft_consensus.cc:504] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.863163 12361 leader_election.cc:288] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131:37721), e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965)
I20260807 00:40:23.866410 12343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.863590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:42287 (local address 127.1.116.131:37721)
0807 00:40:23.863670 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.863672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.863682 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:23.863757 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.863758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.863768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.863826 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:23.863970 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.864461 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.865570 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.866015 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.866103 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.866114 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.866302 (+ 188us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.866343 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:40:23.866423 12362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.863428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:37721 (local address 127.1.116.129:42287)
0807 00:40:23.863573 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.863587 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.863694 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.863831 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.863833 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.863835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:23.863900 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.863902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.864571 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.864572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.865467 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.865470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.866104 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.866105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.866192 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.866195 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.866283 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.866334 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":81,"threads_started":1}
I20260807 00:40:23.866809 12346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.863502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:59537 (local address 127.1.116.130:37965)
0807 00:40:23.863559 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.863562 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.863706 (+ 144us) server_negotiation.cc:407] Connection header received
0807 00:40:23.863801 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.863802 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.863821 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.863887 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:23.864131 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.864664 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.865863 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.866267 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.866364 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.866375 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.866648 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.866732 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:40:23.866799 12364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.863428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:37965 (local address 127.1.116.129:59537)
0807 00:40:23.863606 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:23.863620 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:23.863722 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:23.863892 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:23.863894 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:23.863897 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:23.864043 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.864046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.864750 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.864751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.865724 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:23.865753 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.866379 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.866380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:23.866448 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.866451 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:23.866601 (+ 150us) client_negotiation.cc:240] Negotiation successful
0807 00:40:23.866657 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":3,"thread_start_us":80,"threads_started":1}
I20260807 00:40:23.866990 12239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048362bcb5f24983b8054f64b69e82bd" candidate_uuid: "f3b1ba619ef64d509ea97bf5661c9799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" is_pre_election: true
I20260807 00:40:23.867142 12239 raft_consensus.cc:2454] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3b1ba619ef64d509ea97bf5661c9799 in term 0.
I20260807 00:40:23.867417 11775 leader_election.cc:302] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90eb9bf8a88d4e22974c0a9513b6241a, f3b1ba619ef64d509ea97bf5661c9799; no voters:
I20260807 00:40:23.867656 12361 raft_consensus.cc:2781] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:23.867733 12361 raft_consensus.cc:482] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:23.867787 12361 raft_consensus.cc:3037] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:23.867959 12057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048362bcb5f24983b8054f64b69e82bd" candidate_uuid: "f3b1ba619ef64d509ea97bf5661c9799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e263fdb0d6a848f986ebcce331fc9c45" is_pre_election: true
I20260807 00:40:23.868168 12057 raft_consensus.cc:2454] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3b1ba619ef64d509ea97bf5661c9799 in term 0.
I20260807 00:40:23.868755 12361 raft_consensus.cc:504] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.868918 12361 leader_election.cc:288] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [CANDIDATE]: Term 1 election: Requested vote from peers 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131:37721), e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965)
I20260807 00:40:23.869102 12239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048362bcb5f24983b8054f64b69e82bd" candidate_uuid: "f3b1ba619ef64d509ea97bf5661c9799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90eb9bf8a88d4e22974c0a9513b6241a"
I20260807 00:40:23.869179 12239 raft_consensus.cc:3037] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:23.869822 12239 raft_consensus.cc:2454] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3b1ba619ef64d509ea97bf5661c9799 in term 1.
I20260807 00:40:23.869958 12057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "048362bcb5f24983b8054f64b69e82bd" candidate_uuid: "f3b1ba619ef64d509ea97bf5661c9799" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e263fdb0d6a848f986ebcce331fc9c45"
I20260807 00:40:23.870005 11775 leader_election.cc:302] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90eb9bf8a88d4e22974c0a9513b6241a, f3b1ba619ef64d509ea97bf5661c9799; no voters:
I20260807 00:40:23.870067 12057 raft_consensus.cc:3037] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:23.870141 12361 raft_consensus.cc:2781] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:23.870257 12361 raft_consensus.cc:686] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 1 LEADER]: Becoming Leader. State: Replica: f3b1ba619ef64d509ea97bf5661c9799, State: Running, Role: LEADER
I20260807 00:40:23.870344 12361 consensus_queue.cc:237] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:23.871081 12057 raft_consensus.cc:2454] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3b1ba619ef64d509ea97bf5661c9799 in term 1.
I20260807 00:40:23.871364 10569 catalog_manager.cc:5495] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "f3b1ba619ef64d509ea97bf5661c9799" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } health_report { overall_health: HEALTHY } } }
I20260807 00:40:23.882082 12345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.879550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44920 (local address 127.1.116.129:37929)
0807 00:40:23.879594 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.879596 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.879607 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:40:23.879649 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.879650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.879668 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.879764 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:23.879885 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.880576 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.881381 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.881566 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.881628 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.881823 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:23.881908 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:23.881970 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.882023 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.882039 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:40:23.884143 12239 raft_consensus.cc:1261] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 FOLLOWER]: Refusing update from remote peer f3b1ba619ef64d509ea97bf5661c9799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:23.884233 12057 raft_consensus.cc:1261] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Refusing update from remote peer f3b1ba619ef64d509ea97bf5661c9799: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:23.884604 12366 consensus_queue.cc:1035] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:23.884817 12361 consensus_queue.cc:1035] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:23.888309 12370 mvcc.cc:205] Tried to move back new op lower bound from 7315714965026947072 to 7315714964973531136. Current Snapshot: MvccSnapshot[applied={T|T < 7315714965026947072}]
I20260807 00:40:23.891144 10570 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:23.891254 10570 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60992:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:23.891920 12373 mvcc.cc:205] Tried to move back new op lower bound from 7315714965026947072 to 7315714964973531136. Current Snapshot: MvccSnapshot[applied={T|T < 7315714965026947072}]
I20260807 00:40:23.892802 10570 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c01d75845cb043868f37d813b9b698dc: Sending DeleteTablet for 3 replicas of tablet 048362bcb5f24983b8054f64b69e82bd
I20260807 00:40:23.893188 12211 tablet_service.cc:1514] Processing DeleteTablet for tablet 048362bcb5f24983b8054f64b69e82bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:23 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:49984
I20260807 00:40:23.893229 12028 tablet_service.cc:1514] Processing DeleteTablet for tablet 048362bcb5f24983b8054f64b69e82bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:23 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:55184
I20260807 00:40:23.893632 12382 tablet_replica.cc:330] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: stopping tablet replica
I20260807 00:40:23.893682 11839 tablet_service.cc:1514] Processing DeleteTablet for tablet 048362bcb5f24983b8054f64b69e82bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:23 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:44910
I20260807 00:40:23.893834 12382 raft_consensus.cc:2229] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:23.893991 12382 raft_consensus.cc:2258] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:23.894572 12382 ts_tablet_manager.cc:1905] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:23.894819 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 10525
I20260807 00:40:23.895932 12385 tablet_replica.cc:330] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: stopping tablet replica
I20260807 00:40:23.896091 12385 raft_consensus.cc:2229] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:23.896258 12385 raft_consensus.cc:2258] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:23.896785 12382 ts_tablet_manager.cc:1918] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:23.896919 12382 log.cc:1196] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/048362bcb5f24983b8054f64b69e82bd
I20260807 00:40:23.897271 12382 ts_tablet_manager.cc:1939] T 048362bcb5f24983b8054f64b69e82bd P e263fdb0d6a848f986ebcce331fc9c45: Deleting consensus metadata
I20260807 00:40:23.903172 12383 tablet_replica.cc:330] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: stopping tablet replica
I20260807 00:40:23.903378 12383 raft_consensus.cc:2229] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:23.903551 12383 raft_consensus.cc:2258] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:23.904489 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.894257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48464 (local address 127.1.116.188:43835)
0807 00:40:23.894341 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.894343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.894582 (+ 239us) server_negotiation.cc:407] Connection header received
0807 00:40:23.894691 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.894693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.894710 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.894742 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.896500 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.897609 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.898977 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.899381 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.899431 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.901288 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.901290 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.901291 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.901298 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.903053 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.903481 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.903482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.903483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.903541 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.903751 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.903751 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.903752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.903913 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.903999 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.904236 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.904344 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:40:23.904809 12383 ts_tablet_manager.cc:1905] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:23.906456 12383 ts_tablet_manager.cc:1918] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:23.906562 12383 log.cc:1196] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/048362bcb5f24983b8054f64b69e82bd
I20260807 00:40:23.906844 12383 ts_tablet_manager.cc:1939] T 048362bcb5f24983b8054f64b69e82bd P 90eb9bf8a88d4e22974c0a9513b6241a: Deleting consensus metadata
I20260807 00:40:23.907851 12385 ts_tablet_manager.cc:1905] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:23.909559 12385 ts_tablet_manager.cc:1918] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:23.909638 12385 log.cc:1196] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/048362bcb5f24983b8054f64b69e82bd
I20260807 00:40:23.909904 12385 ts_tablet_manager.cc:1939] T 048362bcb5f24983b8054f64b69e82bd P f3b1ba619ef64d509ea97bf5661c9799: Deleting consensus metadata
W20260807 00:40:23.912284 11778 connection.cc:535] server connection from 127.0.0.1:44910 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 00:40:23.912348 12151 connection.cc:535] server connection from 127.0.0.1:49984 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 00:40:23.912338 11964 connection.cc:535] server connection from 127.0.0.1:55184 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:40:23.922868 12325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.914768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48478 (local address 127.1.116.188:43835)
0807 00:40:23.914819 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.914821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.914874 (+ 53us) server_negotiation.cc:407] Connection header received
0807 00:40:23.914994 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.914996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.915009 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.915069 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.916077 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.916703 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.917523 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.917746 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.917824 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.919434 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.919436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.919436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.919443 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.921327 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.921756 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.921758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.921759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.921823 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.922250 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.922252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.922252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.922403 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.922506 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.922667 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.922751 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 00:40:23.923929 11498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:23.914501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37858 (local address 127.1.116.189:34407)
0807 00:40:23.914577 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:23.914579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:23.914898 (+ 319us) server_negotiation.cc:407] Connection header received
0807 00:40:23.914934 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:23.914934 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:23.914957 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:23.915027 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:23.915958 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.916765 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.917960 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:23.918258 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:23.918298 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:23.920018 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:23.920020 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:23.920021 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:23.920031 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:23.921932 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.922422 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.922423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.922424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.922498 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:23.923382 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:23.923383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:23.923383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:23.923526 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:23.923651 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:23.923739 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 00:40:23.923832 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:40:24.889823 12400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:24.889499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.129:57349)
0807 00:40:24.889657 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:24.889761 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":99,"threads_started":1}
W20260807 00:40:24.890327 11930 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:40717 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:24.890887 12401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:24.890527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.130:41775)
0807 00:40:24.890702 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:24.890814 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
W20260807 00:40:24.891119 12123 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:40717 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:24.891626 12402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:24.891419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.131:38895)
0807 00:40:24.891530 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:24.891593 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":56,"threads_started":1}
W20260807 00:40:24.891860 12300 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:40717 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:25.393980 12406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c01d75845cb043868f37d813b9b698dc)
I20260807 00:40:25.394243 12406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } }
I20260807 00:40:25.394702 12406 leader_election.cc:288] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c01d75845cb043868f37d813b9b698dc (127.1.116.190:40717), 76a774e511124cfc847280da621697c8 (127.1.116.188:43835)
I20260807 00:40:25.395092 12409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.394823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33188)
0807 00:40:25.394966 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.395041 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
W20260807 00:40:25.395285 11402 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:25.395522 12408 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c01d75845cb043868f37d813b9b698dc)
I20260807 00:40:25.395660 12408 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } }
I20260807 00:40:25.395816 12409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.395693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33192)
0807 00:40:25.395753 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.395785 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260807 00:40:25.395893 11402 leader_election.cc:334] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c01d75845cb043868f37d813b9b698dc (127.1.116.190:40717): Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:25.396193 12408 leader_election.cc:288] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c01d75845cb043868f37d813b9b698dc (127.1.116.190:40717), 5fff16d053f545a79859832debbe9cf7 (127.1.116.189:34407)
I20260807 00:40:25.396253 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.396051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33204)
0807 00:40:25.396157 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.396216 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":72,"threads_started":1}
W20260807 00:40:25.396353 11660 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:25.396860 12418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.396619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33212)
0807 00:40:25.396783 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.396826 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
W20260807 00:40:25.397001 11660 leader_election.cc:334] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c01d75845cb043868f37d813b9b698dc (127.1.116.190:40717): Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:25.399098 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.394845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43835 (local address 127.0.0.1:48490)
0807 00:40:25.395049 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.395065 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.395132 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:25.395475 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:25.395478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:25.395481 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:25.395569 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.395571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.396656 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.396657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.397890 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.397892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.398909 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.398910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.398965 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.398969 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:40:25.399015 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 00:40:25.399052 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":48,"threads_started":1}
I20260807 00:40:25.399183 12411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.395054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48490 (local address 127.1.116.188:43835)
0807 00:40:25.395305 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.395308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.395317 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:25.395346 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.395347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.395359 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.395428 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:25.395734 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.396573 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.398011 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.398791 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.398886 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.398901 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.399047 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.399108 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":229,"thread_start_us":108,"threads_started":1}
I20260807 00:40:25.399447 11687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fff16d053f545a79859832debbe9cf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "76a774e511124cfc847280da621697c8" is_pre_election: true
I20260807 00:40:25.399557 11687 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fff16d053f545a79859832debbe9cf7 in term 1.
I20260807 00:40:25.399731 11400 leader_election.cc:302] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fff16d053f545a79859832debbe9cf7, 76a774e511124cfc847280da621697c8; no voters: c01d75845cb043868f37d813b9b698dc
I20260807 00:40:25.399919 12406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 00:40:25.399993 12406 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c01d75845cb043868f37d813b9b698dc)
I20260807 00:40:25.400032 12406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:40:25.400559 12415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.396379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:34407 (local address 127.0.0.1:37868)
0807 00:40:25.396551 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.396563 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.396636 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:25.397278 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:25.397279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:25.397281 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:25.397323 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.397324 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.398169 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.398170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.399533 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.399535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.400263 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.400264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.400347 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.400354 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:40:25.400429 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:40:25.400492 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151}
I20260807 00:40:25.400549 12417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.396428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37868 (local address 127.1.116.189:34407)
0807 00:40:25.396565 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.396567 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.397124 (+ 557us) server_negotiation.cc:407] Connection header received
0807 00:40:25.397150 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.397150 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.397160 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.397248 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:25.397406 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.398063 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.399682 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.400163 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.400258 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.400269 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.400437 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.400488 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260807 00:40:25.400873 11435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "76a774e511124cfc847280da621697c8" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5fff16d053f545a79859832debbe9cf7" is_pre_election: true
I20260807 00:40:25.401338 12406 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } }
I20260807 00:40:25.401491 11435 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 76a774e511124cfc847280da621697c8 in current term 2: Already voted for candidate 5fff16d053f545a79859832debbe9cf7 in this term.
I20260807 00:40:25.401537 12406 leader_election.cc:288] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [CANDIDATE]: Term 2 election: Requested vote from peers c01d75845cb043868f37d813b9b698dc (127.1.116.190:40717), 76a774e511124cfc847280da621697c8 (127.1.116.188:43835)
I20260807 00:40:25.401712 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.401622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33226)
0807 00:40:25.401662 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.401685 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 00:40:25.401723 11687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5fff16d053f545a79859832debbe9cf7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "76a774e511124cfc847280da621697c8"
I20260807 00:40:25.401777 11658 leader_election.cc:302] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 76a774e511124cfc847280da621697c8; no voters: 5fff16d053f545a79859832debbe9cf7, c01d75845cb043868f37d813b9b698dc
I20260807 00:40:25.401827 11687 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:40:25.401974 12409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.401899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33238)
0807 00:40:25.401932 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.401954 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260807 00:40:25.402192 11402 leader_election.cc:334] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c01d75845cb043868f37d813b9b698dc (127.1.116.190:40717): Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
I20260807 00:40:25.402851 11687 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fff16d053f545a79859832debbe9cf7 in term 2.
I20260807 00:40:25.402961 12408 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:40:25.403093 11400 leader_election.cc:302] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5fff16d053f545a79859832debbe9cf7, 76a774e511124cfc847280da621697c8; no voters: c01d75845cb043868f37d813b9b698dc
I20260807 00:40:25.403225 12406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:40:25.403412 12406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [term 2 LEADER]: Becoming Leader. State: Replica: 5fff16d053f545a79859832debbe9cf7, State: Running, Role: LEADER
I20260807 00:40:25.403507 12406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } }
I20260807 00:40:25.403981 12419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fff16d053f545a79859832debbe9cf7. Latest consensus state: current_term: 2 leader_uuid: "5fff16d053f545a79859832debbe9cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:25.404196 12419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:25.404448 12422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:25.405071 12422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:25.405166 12422 catalog_manager.cc:1259] Loaded cluster ID: e4574c42febf434f812f8750e8f37439
I20260807 00:40:25.405207 12422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:25.405351 12422 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:25.405439 12422 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Loaded TSK: 0
I20260807 00:40:25.405580 12422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:40:25.461418 11418 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37858:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:25.463510 11418 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:25.464666 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.464560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33268)
0807 00:40:25.464612 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.464644 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 00:40:25.464720 11687 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [term 2 FOLLOWER]: Refusing update from remote peer 5fff16d053f545a79859832debbe9cf7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 00:40:25.465065 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.464959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.0.0.1:33272)
0807 00:40:25.464997 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.465030 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 00:40:25.465070 12406 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260807 00:40:25.465229 11402 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 -> Peer c01d75845cb043868f37d813b9b698dc (127.1.116.190:40717): Couldn't send request to peer c01d75845cb043868f37d813b9b698dc. Status: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 00:40:25.465854 12408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5fff16d053f545a79859832debbe9cf7. Latest consensus state: current_term: 2 leader_uuid: "5fff16d053f545a79859832debbe9cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:25.465961 12408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:40:25.466651 12406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5fff16d053f545a79859832debbe9cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:25.466743 12406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:25.466719 12408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5fff16d053f545a79859832debbe9cf7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c01d75845cb043868f37d813b9b698dc" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 40717 } } peers { permanent_uuid: "5fff16d053f545a79859832debbe9cf7" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 34407 } attrs { promote: false } } peers { permanent_uuid: "76a774e511124cfc847280da621697c8" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43835 } attrs { promote: false } } }
I20260807 00:40:25.466810 12408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76a774e511124cfc847280da621697c8 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:40:25.472404 12431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.468670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:37965 (local address 127.0.0.1:55200)
0807 00:40:25.468952 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.468964 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.469014 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:25.469598 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:25.469599 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:25.469602 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:25.469643 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.469645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.470626 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.470627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.471393 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.471394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.472174 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.472175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.472247 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.472254 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:40:25.472285 (+ 31us) client_negotiation.cc:240] Negotiation successful
0807 00:40:25.472337 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260807 00:40:25.472532 12410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.468558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:37721 (local address 127.0.0.1:49994)
0807 00:40:25.468599 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.468609 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.468647 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:25.469393 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:25.469394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:25.469396 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:25.469445 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.469446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.470448 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.470449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.471550 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.471553 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.472327 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.472328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.472387 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.472391 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:40:25.472420 (+ 29us) client_negotiation.cc:240] Negotiation successful
0807 00:40:25.472469 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 00:40:25.472708 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.469019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55200 (local address 127.1.116.130:37965)
0807 00:40:25.469451 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.469453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.469459 (+ 6us) server_negotiation.cc:407] Connection header received
0807 00:40:25.469506 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.469507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.469519 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.469593 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:25.469747 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.470488 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.471486 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.472068 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.472518 (+ 450us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.472534 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.472566 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.472611 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":412,"thread_start_us":89,"threads_started":1}
I20260807 00:40:25.472765 12409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.468770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37929 (local address 127.0.0.1:44930)
0807 00:40:25.469016 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.469028 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.469091 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:25.469533 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:25.469535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:25.469540 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:25.469608 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.469610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.470316 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.470317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.471605 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.471607 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.472470 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.472471 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.472577 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.472583 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:40:25.472620 (+ 37us) client_negotiation.cc:240] Negotiation successful
0807 00:40:25.472680 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":5,"thread_start_us":102,"threads_started":1}
I20260807 00:40:25.472767 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.469245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44930 (local address 127.1.116.129:37929)
0807 00:40:25.469376 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.469379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.469388 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:25.469414 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.469415 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.469426 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.469508 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:25.469689 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.470221 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.471808 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.472375 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.472563 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.472598 (+ 35us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.472636 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.472684 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":85,"threads_started":1}
I20260807 00:40:25.472898 12028 tablet_service.cc:1467] Processing CreateTablet for tablet 880dfbcbbde342e48b4774b50f7e7f1c (DEFAULT_TABLE table=test-table [id=fdc16fad7a5e4ed2b810c138a7e921c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:25.473081 12028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880dfbcbbde342e48b4774b50f7e7f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:25.473088 11839 tablet_service.cc:1467] Processing CreateTablet for tablet 880dfbcbbde342e48b4774b50f7e7f1c (DEFAULT_TABLE table=test-table [id=fdc16fad7a5e4ed2b810c138a7e921c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:25.473290 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880dfbcbbde342e48b4774b50f7e7f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:25.473424 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.468658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.1.116.131:37721)
0807 00:40:25.469259 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.469261 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.469277 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:40:25.469303 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.469304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.469315 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.469378 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:25.469553 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.470359 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.471652 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.472229 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.473279 (+ 1050us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.473293 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.473314 (+ 21us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.473353 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":92,"threads_started":1}
I20260807 00:40:25.474342 12435 tablet_bootstrap.cc:492] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: Bootstrap starting.
I20260807 00:40:25.474747 12211 tablet_service.cc:1467] Processing CreateTablet for tablet 880dfbcbbde342e48b4774b50f7e7f1c (DEFAULT_TABLE table=test-table [id=fdc16fad7a5e4ed2b810c138a7e921c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:25.474818 12435 tablet_bootstrap.cc:654] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:25.474884 12211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 880dfbcbbde342e48b4774b50f7e7f1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:25.475158 12437 tablet_bootstrap.cc:492] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799: Bootstrap starting.
I20260807 00:40:25.475346 12435 tablet_bootstrap.cc:492] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: No bootstrap required, opened a new log
I20260807 00:40:25.475405 12435 ts_tablet_manager.cc:1397] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:25.475520 12435 raft_consensus.cc:348] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.475600 12435 raft_consensus.cc:374] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:25.475625 12435 raft_consensus.cc:729] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e263fdb0d6a848f986ebcce331fc9c45, State: Initialized, Role: FOLLOWER
I20260807 00:40:25.475683 12435 consensus_queue.cc:260] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.475701 12437 tablet_bootstrap.cc:654] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:25.475835 12435 ts_tablet_manager.cc:1428] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:25.476426 12444 tablet_bootstrap.cc:492] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Bootstrap starting.
I20260807 00:40:25.476773 12125 heartbeater.cc:503] Master 127.1.116.189:34407 was elected leader, sending a full tablet report...
I20260807 00:40:25.476822 12444 tablet_bootstrap.cc:654] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:25.477051 12437 tablet_bootstrap.cc:492] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799: No bootstrap required, opened a new log
I20260807 00:40:25.477123 12437 ts_tablet_manager.cc:1397] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:25.477293 12437 raft_consensus.cc:348] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.477361 12437 raft_consensus.cc:374] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:25.477382 12437 raft_consensus.cc:729] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3b1ba619ef64d509ea97bf5661c9799, State: Initialized, Role: FOLLOWER
I20260807 00:40:25.477445 12437 consensus_queue.cc:260] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.477636 12437 ts_tablet_manager.cc:1428] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:25.477666 12444 tablet_bootstrap.cc:492] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: No bootstrap required, opened a new log
I20260807 00:40:25.477715 12444 ts_tablet_manager.cc:1397] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:25.477907 12444 raft_consensus.cc:348] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.477989 12444 raft_consensus.cc:374] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:25.478010 12444 raft_consensus.cc:729] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90eb9bf8a88d4e22974c0a9513b6241a, State: Initialized, Role: FOLLOWER
I20260807 00:40:25.478082 12444 consensus_queue.cc:260] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.478257 12444 ts_tablet_manager.cc:1428] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:25.478278 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.477911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.129:44955)
0807 00:40:25.478102 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.478140 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":86,"threads_started":1}
I20260807 00:40:25.478344 12443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.476183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.130:59193)
0807 00:40:25.477899 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.477953 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1683,"thread_start_us":88,"threads_started":1}
I20260807 00:40:25.478945 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.478736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.130:47955)
0807 00:40:25.478867 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.478903 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":88,"threads_started":1}
I20260807 00:40:25.479159 11931 heartbeater.cc:503] Master 127.1.116.189:34407 was elected leader, sending a full tablet report...
I20260807 00:40:25.479226 12301 heartbeater.cc:503] Master 127.1.116.189:34407 was elected leader, sending a full tablet report...
I20260807 00:40:25.479355 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.479260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.129:37359)
0807 00:40:25.479301 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.479327 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260807 00:40:25.479409 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.478612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.131:42275)
0807 00:40:25.479302 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.479350 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":145,"threads_started":1}
I20260807 00:40:25.479771 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.479654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.131:48245)
0807 00:40:25.479714 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.479740 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 00:40:25.667685 12303 tablet.cc:2369] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:25.852057 12448 raft_consensus.cc:482] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:25.852200 12448 raft_consensus.cc:504] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.852595 12448 leader_election.cc:288] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965), f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129:37929)
I20260807 00:40:25.856148 12432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.852985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51459 (local address 127.1.116.130:37965)
0807 00:40:25.853059 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.853062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.853138 (+ 76us) server_negotiation.cc:407] Connection header received
0807 00:40:25.853174 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.853175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.853190 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.853255 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:25.853536 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.854082 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.855311 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.855719 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.855781 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.855789 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.856060 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.856093 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:40:25.856236 12449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.852895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:37965 (local address 127.1.116.131:51459)
0807 00:40:25.852972 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.852987 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.853067 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:25.853307 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:25.853308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:25.853312 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:25.853418 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.853420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.854178 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.854179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.855207 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.855210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.855855 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.855857 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.855951 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.855956 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:25.856037 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:40:25.856090 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":52}
I20260807 00:40:25.856580 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.853046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37929 (local address 127.1.116.131:41173)
0807 00:40:25.853300 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.853314 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.853382 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:25.853537 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:25.853538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:25.853540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:25.853619 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.853621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.854407 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.854408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.855326 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:25.855328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.856255 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.856257 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:25.856343 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.856349 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:40:25.856460 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 00:40:25.856505 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":174,"threads_started":1}
I20260807 00:40:25.856545 12057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880dfbcbbde342e48b4774b50f7e7f1c" candidate_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e263fdb0d6a848f986ebcce331fc9c45" is_pre_election: true
I20260807 00:40:25.856630 12434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.853168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:41173 (local address 127.1.116.129:37929)
0807 00:40:25.853248 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.853251 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.853402 (+ 151us) server_negotiation.cc:407] Connection header received
0807 00:40:25.853439 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.853440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.853452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.853518 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:25.853699 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.854316 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.855421 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.856122 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.856195 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.856208 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.856501 (+ 293us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.856553 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:40:25.856671 12057 raft_consensus.cc:2454] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90eb9bf8a88d4e22974c0a9513b6241a in term 0.
I20260807 00:40:25.856886 12148 leader_election.cc:302] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90eb9bf8a88d4e22974c0a9513b6241a, e263fdb0d6a848f986ebcce331fc9c45; no voters:
I20260807 00:40:25.856952 11865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880dfbcbbde342e48b4774b50f7e7f1c" candidate_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3b1ba619ef64d509ea97bf5661c9799" is_pre_election: true
I20260807 00:40:25.857075 12448 raft_consensus.cc:2781] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:25.857128 12448 raft_consensus.cc:482] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:25.857161 12448 raft_consensus.cc:3037] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:25.857156 11865 raft_consensus.cc:2454] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 90eb9bf8a88d4e22974c0a9513b6241a in term 0.
I20260807 00:40:25.857959 12448 raft_consensus.cc:504] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.858112 12448 leader_election.cc:288] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [CANDIDATE]: Term 1 election: Requested vote from peers e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965), f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129:37929)
I20260807 00:40:25.858367 12057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880dfbcbbde342e48b4774b50f7e7f1c" candidate_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e263fdb0d6a848f986ebcce331fc9c45"
I20260807 00:40:25.858417 11865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "880dfbcbbde342e48b4774b50f7e7f1c" candidate_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3b1ba619ef64d509ea97bf5661c9799"
I20260807 00:40:25.858481 12057 raft_consensus.cc:3037] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:25.858497 11865 raft_consensus.cc:3037] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:25.859248 11865 raft_consensus.cc:2454] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90eb9bf8a88d4e22974c0a9513b6241a in term 1.
I20260807 00:40:25.859373 12057 raft_consensus.cc:2454] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 90eb9bf8a88d4e22974c0a9513b6241a in term 1.
I20260807 00:40:25.859431 12148 leader_election.cc:302] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90eb9bf8a88d4e22974c0a9513b6241a, f3b1ba619ef64d509ea97bf5661c9799; no voters:
I20260807 00:40:25.859529 12448 raft_consensus.cc:2781] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:25.859701 12448 raft_consensus.cc:686] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 LEADER]: Becoming Leader. State: Replica: 90eb9bf8a88d4e22974c0a9513b6241a, State: Running, Role: LEADER
I20260807 00:40:25.859761 12448 consensus_queue.cc:237] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } }
I20260807 00:40:25.860227 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.860073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.131:52589)
0807 00:40:25.860130 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.860164 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 00:40:25.860749 11420 catalog_manager.cc:5495] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a reported cstate change: term changed from 0 to 1, leader changed from <none> to 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131). New cstate: current_term: 1 leader_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90eb9bf8a88d4e22974c0a9513b6241a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 37721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 } health_report { overall_health: UNKNOWN } } }
I20260807 00:40:25.915244 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.912511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49998 (local address 127.1.116.131:37721)
0807 00:40:25.912611 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:25.912614 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:25.912625 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:40:25.912692 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:25.912693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:25.912714 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:25.912793 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:25.912962 (+ 169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.913548 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.914328 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:25.914572 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:25.914664 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:25.914821 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:25.914977 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:25.915067 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:25.915120 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 00:40:25.915142 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:40:25.916899 12057 raft_consensus.cc:1261] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Refusing update from remote peer 90eb9bf8a88d4e22974c0a9513b6241a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:25.916998 11865 raft_consensus.cc:1261] T 880dfbcbbde342e48b4774b50f7e7f1c P f3b1ba619ef64d509ea97bf5661c9799 [term 1 FOLLOWER]: Refusing update from remote peer 90eb9bf8a88d4e22974c0a9513b6241a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:25.917310 12452 consensus_queue.cc:1035] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e263fdb0d6a848f986ebcce331fc9c45" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:25.917443 12448 consensus_queue.cc:1035] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3b1ba619ef64d509ea97bf5661c9799" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:25.918512 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.918335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.129:46537)
0807 00:40:25.918419 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.918475 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 00:40:25.918787 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.918673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.130:45643)
0807 00:40:25.918727 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.918753 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 00:40:25.920148 12451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.920041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.131:58591)
0807 00:40:25.920090 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.920114 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 00:40:25.920859 12459 mvcc.cc:205] Tried to move back new op lower bound from 7315714973352628224 to 7315714973122203648. Current Snapshot: MvccSnapshot[applied={T|T < 7315714973352628224}]
I20260807 00:40:25.920976 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.920868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.129:41931)
0807 00:40:25.920916 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.920953 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 00:40:25.922905 12450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:25.922787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:40717 (local address 127.1.116.130:60103)
0807 00:40:25.922835 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.922863 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:40717: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260807 00:40:25.923138 11420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:25.923233 11420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37858:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:25.924765 11420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5fff16d053f545a79859832debbe9cf7: Sending DeleteTablet for 3 replicas of tablet 880dfbcbbde342e48b4774b50f7e7f1c
I20260807 00:40:25.925058 12211 tablet_service.cc:1514] Processing DeleteTablet for tablet 880dfbcbbde342e48b4774b50f7e7f1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:25 UTC) from {username='oryx', principal='oryx/127.1.116.189@KRBTEST.COM'} at 127.0.0.1:49994
I20260807 00:40:25.925546 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 11628
I20260807 00:40:25.925590 12028 tablet_service.cc:1514] Processing DeleteTablet for tablet 880dfbcbbde342e48b4774b50f7e7f1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:25 UTC) from {username='oryx', principal='oryx/127.1.116.189@KRBTEST.COM'} at 127.0.0.1:55200
I20260807 00:40:25.925631 11839 tablet_service.cc:1514] Processing DeleteTablet for tablet 880dfbcbbde342e48b4774b50f7e7f1c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:25 UTC) from {username='oryx', principal='oryx/127.1.116.189@KRBTEST.COM'} at 127.0.0.1:44930
I20260807 00:40:25.936048 12469 tablet_replica.cc:330] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: stopping tablet replica
I20260807 00:40:25.936113 12469 raft_consensus.cc:2229] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:25.936203 12469 raft_consensus.cc:2258] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:25.937394 12469 ts_tablet_manager.cc:1905] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:25.938598 12470 tablet_replica.cc:330] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: stopping tablet replica
I20260807 00:40:25.938663 12470 raft_consensus.cc:2229] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:25.938711 12470 raft_consensus.cc:2258] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:25.939464 12470 ts_tablet_manager.cc:1905] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:25.940769 12469 ts_tablet_manager.cc:1918] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:25.940828 12469 log.cc:1196] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/880dfbcbbde342e48b4774b50f7e7f1c
I20260807 00:40:25.941059 12469 ts_tablet_manager.cc:1939] T 880dfbcbbde342e48b4774b50f7e7f1c P 90eb9bf8a88d4e22974c0a9513b6241a: Deleting consensus metadata
I20260807 00:40:25.941399 11401 catalog_manager.cc:4841] TS 90eb9bf8a88d4e22974c0a9513b6241a (127.1.116.131:37721): tablet 880dfbcbbde342e48b4774b50f7e7f1c (table test-table [id=fdc16fad7a5e4ed2b810c138a7e921c0]) successfully deleted
I20260807 00:40:25.941901 12470 ts_tablet_manager.cc:1918] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:25.941962 12470 log.cc:1196] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/880dfbcbbde342e48b4774b50f7e7f1c
I20260807 00:40:25.942153 12470 ts_tablet_manager.cc:1939] T 880dfbcbbde342e48b4774b50f7e7f1c P e263fdb0d6a848f986ebcce331fc9c45: Deleting consensus metadata
I20260807 00:40:25.942610 11400 catalog_manager.cc:4841] TS e263fdb0d6a848f986ebcce331fc9c45 (127.1.116.130:37965): tablet 880dfbcbbde342e48b4774b50f7e7f1c (table test-table [id=fdc16fad7a5e4ed2b810c138a7e921c0]) successfully deleted
W20260807 00:40:25.950402 12409 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:25.950161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37929 (local address 127.0.0.1:44932)
0807 00:40:25.950244 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:25.950260 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:25.950329 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37929: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":55}
W20260807 00:40:25.950614 11400 catalog_manager.cc:4584] TS f3b1ba619ef64d509ea97bf5661c9799 (127.1.116.129:37929): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 880dfbcbbde342e48b4774b50f7e7f1c: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37929: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260807 00:40:25.950899 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 11937
I20260807 00:40:25.967085 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 12130
I20260807 00:40:25.976913 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 11244
I20260807 00:40:25.990216 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 11503
2026-08-07T00:40:25Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5510 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:26 dist-test-slave-h6n0 krb5kdc[12479](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:26 dist-test-slave-h6n0 krb5kdc[12479](info): set up 2 sockets
Aug 07 00:40:26 dist-test-slave-h6n0 krb5kdc[12479](info): commencing operation
krb5kdc: starting...
W20260807 00:40:27.224929 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.208s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:27 dist-test-slave-h6n0 krb5kdc[12479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:27Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab.
Entry for principal oryx/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab.
I20260807 00:40:27.353832 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35543
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab
--principal=oryx/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:35543
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:27.445175 12499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:27.445411 12499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:27.445433 12499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:27.446624 12499 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:27.446669 12499 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 00:40:27.446683 12499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:27.446697 12499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:27.446709 12499 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:27.446722 12499 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:27.448318 12499 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:35543
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.190.keytab
--principal=oryx/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:27.448552 12499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:27.448760 12499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:27.454356 12505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:27.454349 12507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:27.454371 12509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:27.454602 12499 server_base.cc:1034] running on GCE node
I20260807 00:40:27.455015 12499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:27.455401 12499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:27.456601 12499 hybrid_clock.cc:648] HybridClock initialized: now 1786063227456570 us; error 58 us; skew 500 ppm
Aug 07 00:40:27 dist-test-slave-h6n0 krb5kdc[12479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:27.464479 12499 init.cc:377] Logged in from keytab as oryx/127.1.116.190@KRBTEST.COM (short username oryx)
I20260807 00:40:27.465042 12499 webserver.cc:460] Webserver started at http://127.1.116.190:43743/ using document root <none> and password file <none>
I20260807 00:40:27.465306 12499 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:27.465378 12499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:27.465539 12499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:27.466835 12499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "c9318c034b6a4d34bbb8fa9651720e2f"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "09fb32bb863c59adbd94be56699aa8c6"
server_key_iv: "0b9c62d22d143a1b3d83c346897e5c49"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.467295 12499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "c9318c034b6a4d34bbb8fa9651720e2f"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "09fb32bb863c59adbd94be56699aa8c6"
server_key_iv: "0b9c62d22d143a1b3d83c346897e5c49"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.468827 12499 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:27.469717 12519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:27.469970 12499 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:27.470090 12499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "c9318c034b6a4d34bbb8fa9651720e2f"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "09fb32bb863c59adbd94be56699aa8c6"
server_key_iv: "0b9c62d22d143a1b3d83c346897e5c49"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.470189 12499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:27.487262 12499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:27.489713 12499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:27.489888 12499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:27.496517 12499 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:35543
I20260807 00:40:27.496649 12604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:35543 every 8 connection(s)
I20260807 00:40:27.497018 12499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:27.497988 12606 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:27.499696 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 12499
I20260807 00:40:27.499796 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:27.500046 1490 external_mini_cluster.cc:1428] Setting key 23d11891ac16738797be947c43b082ec
I20260807 00:40:27.501618 12606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Bootstrap starting.
I20260807 00:40:27.502354 12606 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:27.502758 12606 log.cc:824] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:27.503557 12606 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: No bootstrap required, opened a new log
I20260807 00:40:27.505683 12606 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35543 } }
I20260807 00:40:27.505863 12606 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:27.505916 12606 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9318c034b6a4d34bbb8fa9651720e2f, State: Initialized, Role: FOLLOWER
I20260807 00:40:27.506083 12606 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35543 } }
I20260807 00:40:27.506186 12606 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:27.506230 12606 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:27.506281 12606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:27.507392 12606 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35543 } }
Aug 07 00:40:27 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:27.507534 12606 leader_election.cc:302] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9318c034b6a4d34bbb8fa9651720e2f; no voters:
I20260807 00:40:27.507773 12606 leader_election.cc:288] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:27.507804 12614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:27.507951 12614 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [term 1 LEADER]: Becoming Leader. State: Replica: c9318c034b6a4d34bbb8fa9651720e2f, State: Running, Role: LEADER
I20260807 00:40:27.508109 12606 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:27.508065 12614 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35543 } }
I20260807 00:40:27.510342 12615 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9318c034b6a4d34bbb8fa9651720e2f. Latest consensus state: current_term: 1 leader_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35543 } } }
I20260807 00:40:27.510555 12616 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9318c034b6a4d34bbb8fa9651720e2f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35543 } } }
W20260807 00:40:27.510768 12630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260807 00:40:27.510838 12616 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:27.510780 12615 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f [sys.catalog]: This master's current role is: LEADER
W20260807 00:40:27.510852 12630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:27.511507 12636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:27.511760 12636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:27.512797 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.500709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50350 (local address 127.1.116.190:35543)
0807 00:40:27.500928 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:27.500939 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:27.500978 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:40:27.501530 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:27.501538 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:27.501591 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:27.501768 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:27.502461 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.503570 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.504343 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.504808 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.504937 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.508223 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:27.508229 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:27.508229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:27.508246 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:27.510354 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.511042 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.511046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.511047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.511217 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.511481 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.511482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.511482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.511957 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:27.512118 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:27.512355 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 00:40:27.512565 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":106,"threads_started":1}
I20260807 00:40:27.513589 12636 catalog_manager.cc:1347] Generated new cluster ID: 2e36121f1f1c490b8d272729ca7f5cbe
I20260807 00:40:27.513652 12636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:27.524945 12636 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:27.525575 12636 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:27.528455 12636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Generated new TSK 0
I20260807 00:40:27.528651 12636 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab.
Entry for principal oryx/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab.
I20260807 00:40:27.563835 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35543
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab
--principal=oryx/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:27.667344 12640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:27.667649 12640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:27.667712 12640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:27.668957 12640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:27.669001 12640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:27.669023 12640 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:40:27.669041 12640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:27.670728 12640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab
--principal=oryx/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:27.671016 12640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:27.671291 12640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:27.676887 12647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:27.676887 12649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:27.676918 12646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:27.677410 12640 server_base.cc:1034] running on GCE node
I20260807 00:40:27.677649 12640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:27.677906 12640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:27.679085 12640 hybrid_clock.cc:648] HybridClock initialized: now 1786063227679076 us; error 34 us; skew 500 ppm
Aug 07 00:40:27 dist-test-slave-h6n0 krb5kdc[12479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:27.682482 12640 init.cc:377] Logged in from keytab as oryx/127.1.116.129@KRBTEST.COM (short username oryx)
I20260807 00:40:27.683035 12640 webserver.cc:460] Webserver started at http://127.1.116.129:44745/ using document root <none> and password file <none>
I20260807 00:40:27.683336 12640 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:27.683411 12640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:27.683570 12640 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:27.684763 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "54674934aed84cb0a147040afdb5aea9"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "b228a1fb6a9e44c67c185333edc92519"
server_key_iv: "9b9c3b7fd2ba047909d106e01b5f049b"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.685159 12640 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "54674934aed84cb0a147040afdb5aea9"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "b228a1fb6a9e44c67c185333edc92519"
server_key_iv: "9b9c3b7fd2ba047909d106e01b5f049b"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.686678 12640 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 00:40:27.687559 12660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:27.687831 12640 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:27.687951 12640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "54674934aed84cb0a147040afdb5aea9"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "b228a1fb6a9e44c67c185333edc92519"
server_key_iv: "9b9c3b7fd2ba047909d106e01b5f049b"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.688031 12640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:27.706679 12640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:27.709323 12640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:27.709530 12640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:27.710192 12640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:27.710242 12640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:27.710268 12640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:27.710280 12640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:40:27 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:27.719173 12640 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:44769
I20260807 00:40:27.719291 12820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:44769 every 8 connection(s)
I20260807 00:40:27.719564 12640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:27.720856 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 12640
I20260807 00:40:27.721002 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:27.721242 1490 external_mini_cluster.cc:1428] Setting key 98028bd140b46eec56327919c7e30f33
I20260807 00:40:27.721824 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.710801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:38211 (local address 127.1.116.190:35543)
0807 00:40:27.710853 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:27.710856 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:27.711194 (+ 338us) server_negotiation.cc:407] Connection header received
0807 00:40:27.711877 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:27.711880 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:27.711901 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:27.711994 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:27.713320 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.714216 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.715508 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.715758 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.715879 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.718651 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:27.718652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:27.718652 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:27.718658 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:27.720255 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.720686 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.720687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.720688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.720753 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.721056 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.721058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.721058 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.721216 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:27.721305 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:27.721607 (+ 302us) server_negotiation.cc:299] Negotiation successful
0807 00:40:27.721723 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:40:27.722013 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.710822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35543 (local address 127.1.116.129:38211)
0807 00:40:27.711047 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:27.711090 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:27.711691 (+ 601us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:27.712207 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:27.712214 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:27.712831 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:27.713158 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.713166 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.714354 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.714355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.715400 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.715403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.715891 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.715892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.716056 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.716678 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:27.716688 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:27.718494 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:27.720341 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.720346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.720348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.720587 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.720832 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.720832 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.720833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.720933 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.721301 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:27.721310 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:27.721462 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 00:40:27.721581 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 00:40:27.721774 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":109,"threads_started":1}
I20260807 00:40:27.722702 12821 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35543
I20260807 00:40:27.722795 12821 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:27.722951 12821 heartbeater.cc:511] Master 127.1.116.190:35543 requested a full tablet report, sending...
I20260807 00:40:27.723570 12541 ts_manager.cc:194] Registered new tserver with Master: 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769)
I20260807 00:40:27.724529 12541 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:38211
I20260807 00:40:27.730239 12829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.724035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35543 (local address 127.1.116.129:53029)
0807 00:40:27.724176 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:27.724186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:27.724269 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:27.724402 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:27.724403 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:27.724592 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:27.724685 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.724687 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.725339 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.725340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.725906 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.725908 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.726359 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.726360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.726432 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.726980 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:27.726986 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:27.727309 (+ 323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:27.729196 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.729197 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.729197 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.729368 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.729608 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.729609 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.729609 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.729663 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.729981 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:27.729981 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:27.730020 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 00:40:27.730069 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 00:40:27.730173 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260807 00:40:27.730325 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.724074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:53029 (local address 127.1.116.190:35543)
0807 00:40:27.724122 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:27.724124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:27.724254 (+ 130us) server_negotiation.cc:407] Connection header received
0807 00:40:27.724328 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:27.724329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:27.724340 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:27.724396 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:27.724752 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.725234 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.725992 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.726275 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.726346 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.727412 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:27.727414 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:27.727414 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:27.727422 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:27.729096 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.729451 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.729452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.729453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.729517 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.729739 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.729740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.729740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.729907 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:27.729986 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:27.730118 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 00:40:27.730198 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
WARNING: no policy specified for oryx/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab.
Entry for principal oryx/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab.
I20260807 00:40:27.753067 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35543
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab
--principal=oryx/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:27.849573 12833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:27.849850 12833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:27.849895 12833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:27.851193 12833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:27.851258 12833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:27.851296 12833 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:40:27.851354 12833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:27.852914 12833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab
--principal=oryx/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:27.853214 12833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:27.853468 12833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:27.859362 12839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:27.859320 12842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:27.859319 12840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:27.859530 12833 server_base.cc:1034] running on GCE node
I20260807 00:40:27.859856 12833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:27.860204 12833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:27.861382 12833 hybrid_clock.cc:648] HybridClock initialized: now 1786063227861363 us; error 37 us; skew 500 ppm
Aug 07 00:40:27 dist-test-slave-h6n0 krb5kdc[12479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:27.864374 12833 init.cc:377] Logged in from keytab as oryx/127.1.116.130@KRBTEST.COM (short username oryx)
I20260807 00:40:27.864919 12833 webserver.cc:460] Webserver started at http://127.1.116.130:32949/ using document root <none> and password file <none>
I20260807 00:40:27.865180 12833 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:27.865231 12833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:27.865391 12833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:27.866796 12833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "7bb0e5e5020f4124b11df2cc73874b96"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "69a5712558c9f77e50761190f8974ee3"
server_key_iv: "4eec1ef5ae125615465edbc9cf2edbc6"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.867197 12833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "7bb0e5e5020f4124b11df2cc73874b96"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "69a5712558c9f77e50761190f8974ee3"
server_key_iv: "4eec1ef5ae125615465edbc9cf2edbc6"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.868821 12833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:27.869772 12855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:27.870072 12833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:27.870172 12833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "7bb0e5e5020f4124b11df2cc73874b96"
format_stamp: "Formatted at 2026-08-07 00:40:27 on dist-test-slave-h6n0"
server_key: "69a5712558c9f77e50761190f8974ee3"
server_key_iv: "4eec1ef5ae125615465edbc9cf2edbc6"
server_key_version: "encryptionkey@0"
I20260807 00:40:27.870273 12833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:27.883630 12833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:27.885991 12833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:27.886152 12833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:27.887010 12833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:27.887058 12833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:27.887079 12833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:27.887094 12833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:40:27 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:27.895257 12833 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:33313
I20260807 00:40:27.895414 13025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:33313 every 8 connection(s)
I20260807 00:40:27.895697 12833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:27.898422 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.887541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:56029 (local address 127.1.116.190:35543)
0807 00:40:27.887606 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:27.887609 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:27.887852 (+ 243us) server_negotiation.cc:407] Connection header received
0807 00:40:27.888637 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:27.888639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:27.888657 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:27.888738 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:27.889789 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.890657 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.891983 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.892301 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.892388 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.895344 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:27.895346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:27.895346 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:27.895356 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:27.896848 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.897338 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.897339 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.897340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.897402 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.897695 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.897697 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.897697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.897866 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:27.897944 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:27.898269 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 00:40:27.898355 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:40:27.898598 12868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.887478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35543 (local address 127.1.116.130:56029)
0807 00:40:27.887702 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:27.887746 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:27.888466 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:27.888842 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:27.888848 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:27.889338 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:27.889687 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.889694 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.890778 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.890780 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.891879 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.891882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.892412 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.892413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.892567 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.893166 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:27.893176 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:27.895199 (+ 2023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:27.896932 (+ 1733us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.896939 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.896941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.897230 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.897484 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.897485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.897485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.897589 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.897939 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:27.897944 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:27.898068 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 00:40:27.898193 (+ 125us) client_negotiation.cc:240] Negotiation successful
0807 00:40:27.898335 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":90,"threads_started":1}
I20260807 00:40:27.899349 13026 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35543
I20260807 00:40:27.899471 13026 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:27.899708 13026 heartbeater.cc:511] Master 127.1.116.190:35543 requested a full tablet report, sending...
I20260807 00:40:27.899739 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 12833
I20260807 00:40:27.899864 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:27.900092 1490 external_mini_cluster.cc:1428] Setting key 438f5b0f72e3dd547a5c3bbad2bd64c9
I20260807 00:40:27.900593 12541 ts_manager.cc:194] Registered new tserver with Master: 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313)
I20260807 00:40:27.901106 12541 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:56029
I20260807 00:40:27.909499 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.900738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:43837 (local address 127.1.116.190:35543)
0807 00:40:27.900781 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:27.900783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:27.902695 (+ 1912us) server_negotiation.cc:407] Connection header received
0807 00:40:27.902755 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:27.902756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:27.902766 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:27.902822 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:27.903324 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.904126 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.904995 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.905186 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.905247 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.906433 (+ 1186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:27.906434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:27.906434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:27.906440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:27.908036 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.908563 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.908564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.908564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.908615 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:27.908880 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:27.908881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:27.908881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:27.909037 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:27.909099 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:27.909377 (+ 278us) server_negotiation.cc:299] Negotiation successful
0807 00:40:27.909439 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 00:40:27.909544 13034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:27.900787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35543 (local address 127.1.116.130:43837)
0807 00:40:27.900960 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:27.902609 (+ 1649us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:27.902692 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:27.902852 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:27.902852 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:27.903121 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:27.903221 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.903223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.904271 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.904272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.904895 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:27.904897 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:27.905275 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:27.905276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:27.905357 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:27.905873 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:27.905879 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:27.906325 (+ 446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:27.908201 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.908202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.908202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.908471 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.908689 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:27.908690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:27.908691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:27.908797 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:27.909119 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:27.909121 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:27.909187 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 00:40:27.909299 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 00:40:27.909419 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for oryx/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab.
Entry for principal oryx/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab.
I20260807 00:40:27.931133 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35543
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab
--principal=oryx/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:34983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:28.019645 13039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:28.019862 13039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:28.019903 13039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:28.021000 13039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:28.021067 13039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:28.021104 13039 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:40:28.021159 13039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:28.022732 13039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:34983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab
--principal=oryx/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.116.190:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:28.022975 13039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:28.023244 13039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:28.028148 13044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:28.028332 13047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:28.028165 13045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:28.028589 13039 server_base.cc:1034] running on GCE node
I20260807 00:40:28.029083 13039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:28.029422 13039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:28.030587 13039 hybrid_clock.cc:648] HybridClock initialized: now 1786063228030559 us; error 42 us; skew 500 ppm
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063228, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:28.034310 13039 init.cc:377] Logged in from keytab as oryx/127.1.116.131@KRBTEST.COM (short username oryx)
I20260807 00:40:28.034901 13039 webserver.cc:460] Webserver started at http://127.1.116.131:46281/ using document root <none> and password file <none>
I20260807 00:40:28.035161 13039 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:28.035238 13039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:28.035401 13039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:28.036573 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "688dd3db2c584419a495ac81582e0afb"
format_stamp: "Formatted at 2026-08-07 00:40:28 on dist-test-slave-h6n0"
server_key: "e02da418f8f0ff2daeeb4a4c28bc7653"
server_key_iv: "a4c288c49844c91f67f9265e1e5e98b8"
server_key_version: "encryptionkey@0"
I20260807 00:40:28.036979 13039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "688dd3db2c584419a495ac81582e0afb"
format_stamp: "Formatted at 2026-08-07 00:40:28 on dist-test-slave-h6n0"
server_key: "e02da418f8f0ff2daeeb4a4c28bc7653"
server_key_iv: "a4c288c49844c91f67f9265e1e5e98b8"
server_key_version: "encryptionkey@0"
I20260807 00:40:28.038475 13039 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:28.039388 13055 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:28.039628 13039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:28.039752 13039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "688dd3db2c584419a495ac81582e0afb"
format_stamp: "Formatted at 2026-08-07 00:40:28 on dist-test-slave-h6n0"
server_key: "e02da418f8f0ff2daeeb4a4c28bc7653"
server_key_iv: "a4c288c49844c91f67f9265e1e5e98b8"
server_key_version: "encryptionkey@0"
I20260807 00:40:28.039829 13039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:28.063246 13039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:28.065904 13039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:28.066082 13039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:28.066856 13039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:28.066920 13039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:28.066968 13039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:28.066990 13039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063228, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:28.076890 13039 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44075
I20260807 00:40:28.076992 13221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44075 every 8 connection(s)
I20260807 00:40:28.077426 13039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:28.078038 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 13039
I20260807 00:40:28.078145 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:28.078395 1490 external_mini_cluster.cc:1428] Setting key ca078e32d2dad50784c1606602965c79
I20260807 00:40:28.079879 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.067407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:59545 (local address 127.1.116.190:35543)
0807 00:40:28.067500 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.067503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.067863 (+ 360us) server_negotiation.cc:407] Connection header received
0807 00:40:28.068909 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.068910 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.068922 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.068984 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:28.070163 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.070837 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.071928 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.072123 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.072203 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.076083 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:28.076084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:28.076085 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:28.076092 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:28.077992 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.078599 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.078602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.078602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.078696 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.079096 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.079097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.079098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.079286 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:28.079376 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.079703 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.079801 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:40:28.080003 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.067372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35543 (local address 127.1.116.131:59545)
0807 00:40:28.067620 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.067697 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.068736 (+ 1039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.069117 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.069124 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.069645 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:28.070037 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.070044 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.070951 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.070952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.071814 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.071817 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.072215 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.072216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.072348 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.072906 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:28.072920 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:28.075980 (+ 3060us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:28.078084 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:28.078087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:28.078089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:28.078468 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:28.078826 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:28.078828 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:28.078828 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:28.079001 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:28.079379 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:28.079385 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:28.079538 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.079656 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.079808 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":121,"threads_started":1}
I20260807 00:40:28.080613 13222 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35543
I20260807 00:40:28.080708 13222 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:28.080925 13222 heartbeater.cc:511] Master 127.1.116.190:35543 requested a full tablet report, sending...
I20260807 00:40:28.081669 12541 ts_manager.cc:194] Registered new tserver with Master: 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075)
I20260807 00:40:28.082175 12541 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:59545
I20260807 00:40:28.087749 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.081681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35543 (local address 127.1.116.131:55481)
0807 00:40:28.081890 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.081903 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.081975 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.082107 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.082107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.082413 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:28.082484 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.082486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.083514 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.083515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.084022 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.084023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.084394 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.084394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.084440 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.084815 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:28.084819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:28.085078 (+ 259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:28.086935 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:28.086936 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:28.086936 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:28.087088 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:28.087258 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:28.087258 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:28.087259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:28.087306 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:28.087512 (+ 206us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:28.087513 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:28.087548 (+ 35us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.087611 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.087683 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":109,"threads_started":1}
I20260807 00:40:28.087764 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.081722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:55481 (local address 127.1.116.190:35543)
0807 00:40:28.081759 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.081761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.081984 (+ 223us) server_negotiation.cc:407] Connection header received
0807 00:40:28.082034 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.082035 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.082045 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.082102 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:28.082598 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.083385 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.084109 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.084311 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.084372 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.085168 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:28.085169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:28.085169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:28.085176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:28.086859 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.087149 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.087150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.087150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.087197 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.087353 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.087354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.087354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.087447 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:28.087501 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.087639 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.087697 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:40:28.090601 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063228, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:40:28.116920 12611 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:28.110499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.1.116.190:35543)
0807 00:40:28.110573 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.110576 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.110721 (+ 145us) server_negotiation.cc:407] Connection header received
0807 00:40:28.110752 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.110753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.110765 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.110818 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:28.111666 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.112230 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.113012 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.113236 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.113313 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.116810 (+ 3497us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50362: BlockingRecv error: recv got EOF from 127.0.0.1:50362 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063228, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:28.125146 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.117748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.1.116.190:35543)
0807 00:40:28.117838 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.117840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.117951 (+ 111us) server_negotiation.cc:407] Connection header received
0807 00:40:28.117982 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.117983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.117997 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.118055 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:28.118649 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.119124 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.119869 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.120054 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.120123 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.122482 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:28.122483 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:28.122484 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:28.122490 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:28.123913 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.124304 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.124304 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.124305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.124357 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.124557 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.124557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.124557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.124654 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:28.124737 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.124897 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.125004 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:40:28.126981 12541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:28.128511 12541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:28.136556 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.132104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.1.116.131:44075)
0807 00:40:28.132343 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.132349 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.132668 (+ 319us) server_negotiation.cc:407] Connection header received
0807 00:40:28.132716 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.132718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.132734 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.132811 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:28.133005 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.133755 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.135448 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.135973 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.136074 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.136153 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.136358 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.136415 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":117,"threads_started":1}
I20260807 00:40:28.136590 13261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.131749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:33313 (local address 127.0.0.1:37676)
0807 00:40:28.131959 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.132004 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.132079 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.132631 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.132635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.132638 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:28.132714 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.132716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.134051 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.134051 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.135343 (+ 1292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.135345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.136176 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.136177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.136242 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.136248 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.136325 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.136391 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":10,"thread_start_us":151,"threads_started":1}
I20260807 00:40:28.136502 13262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.131808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44075 (local address 127.0.0.1:56638)
0807 00:40:28.132545 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.132555 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.132610 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.132877 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.132878 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.132880 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:28.132915 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.132916 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.133854 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.133855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.135341 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.135345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.136077 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.136078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.136181 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.136200 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.136283 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.136369 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"mutex_wait_us":11,"thread_start_us":129,"threads_started":1}
I20260807 00:40:28.136722 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.132033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37676 (local address 127.1.116.130:33313)
0807 00:40:28.132457 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.132461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.132472 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:40:28.132518 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.132520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.132539 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.132625 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:28.132802 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.133947 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.135491 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.136061 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.136172 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.136272 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.136540 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.136600 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":98,"threads_started":1}
I20260807 00:40:28.137787 13129 tablet_service.cc:1467] Processing CreateTablet for tablet 15020f8656614a64882b0ef9ba0f554d (DEFAULT_TABLE table=test-table [id=5e9ef4e4a4f24d8da786c62fddf52784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.137753 12929 tablet_service.cc:1467] Processing CreateTablet for tablet 15020f8656614a64882b0ef9ba0f554d (DEFAULT_TABLE table=test-table [id=5e9ef4e4a4f24d8da786c62fddf52784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.137926 13263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.131748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44769 (local address 127.0.0.1:54808)
0807 00:40:28.132338 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.132352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.132436 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.132725 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.132727 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.132730 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:28.132816 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.132818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.134109 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.134111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.135271 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.135275 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.137648 (+ 2373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.137650 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.137724 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.137729 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.137788 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.137853 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"mutex_wait_us":8,"thread_start_us":86,"threads_started":1}
I20260807 00:40:28.138260 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15020f8656614a64882b0ef9ba0f554d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.138341 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15020f8656614a64882b0ef9ba0f554d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.138469 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.132047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.1.116.129:44769)
0807 00:40:28.132318 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.132333 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.132425 (+ 92us) server_negotiation.cc:407] Connection header received
0807 00:40:28.132520 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.132524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.132546 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.132643 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:28.132923 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.134029 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.136199 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.137423 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.137519 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.137603 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.138268 (+ 665us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.138347 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":151,"threads_started":1}
I20260807 00:40:28.141626 13269 tablet_bootstrap.cc:492] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Bootstrap starting.
I20260807 00:40:28.142335 12725 tablet_service.cc:1467] Processing CreateTablet for tablet 15020f8656614a64882b0ef9ba0f554d (DEFAULT_TABLE table=test-table [id=5e9ef4e4a4f24d8da786c62fddf52784]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.142511 13269 tablet_bootstrap.cc:654] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.142796 12725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 15020f8656614a64882b0ef9ba0f554d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.142918 13269 log.cc:824] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:28.143224 13270 tablet_bootstrap.cc:492] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Bootstrap starting.
I20260807 00:40:28.143739 13269 tablet_bootstrap.cc:492] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: No bootstrap required, opened a new log
I20260807 00:40:28.143836 13269 ts_tablet_manager.cc:1397] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 00:40:28.144026 13270 tablet_bootstrap.cc:654] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.144358 13270 log.cc:824] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:28.145128 13270 tablet_bootstrap.cc:492] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: No bootstrap required, opened a new log
I20260807 00:40:28.145215 13270 ts_tablet_manager.cc:1397] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:28.146188 13269 raft_consensus.cc:348] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.146377 13269 raft_consensus.cc:374] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.146428 13269 raft_consensus.cc:729] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688dd3db2c584419a495ac81582e0afb, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.146627 13269 consensus_queue.cc:260] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.146972 13269 ts_tablet_manager.cc:1428] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:40:28.147015 13222 heartbeater.cc:503] Master 127.1.116.190:35543 was elected leader, sending a full tablet report...
I20260807 00:40:28.146961 13270 raft_consensus.cc:348] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.147185 13270 raft_consensus.cc:374] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.147239 13270 raft_consensus.cc:729] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb0e5e5020f4124b11df2cc73874b96, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.147367 13270 consensus_queue.cc:260] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.147658 13270 ts_tablet_manager.cc:1428] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 00:40:28.147804 13026 heartbeater.cc:503] Master 127.1.116.190:35543 was elected leader, sending a full tablet report...
I20260807 00:40:28.148299 13279 tablet_bootstrap.cc:492] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Bootstrap starting.
I20260807 00:40:28.148972 13279 tablet_bootstrap.cc:654] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.149325 13279 log.cc:824] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:28.150147 13279 tablet_bootstrap.cc:492] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: No bootstrap required, opened a new log
I20260807 00:40:28.150236 13279 ts_tablet_manager.cc:1397] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 00:40:28.151839 13279 raft_consensus.cc:348] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.151995 13279 raft_consensus.cc:374] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.152043 13279 raft_consensus.cc:729] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.152189 13279 consensus_queue.cc:260] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.152494 13279 ts_tablet_manager.cc:1428] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 00:40:28.152570 12821 heartbeater.cc:503] Master 127.1.116.190:35543 was elected leader, sending a full tablet report...
I20260807 00:40:28.159165 13283 raft_consensus.cc:482] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:28.159260 13283 raft_consensus.cc:504] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.159618 13283 leader_election.cc:288] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313), 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075)
I20260807 00:40:28.162906 12671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.159768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:33313 (local address 127.1.116.129:41707)
0807 00:40:28.159830 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.159843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.159889 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.160153 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.160154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.160156 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:28.160231 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.160233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.161050 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.161052 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.161881 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.161884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.162573 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.162574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.162688 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.162693 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.162758 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.162798 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260807 00:40:28.162919 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.159887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:41707 (local address 127.1.116.130:33313)
0807 00:40:28.159968 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.159971 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.159982 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:40:28.160013 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.160014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.160024 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.160105 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:28.160331 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.160954 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.161988 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.162440 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.162509 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.162519 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.162779 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.162838 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:40:28.162977 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.160067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:32819 (local address 127.1.116.131:44075)
0807 00:40:28.160141 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.160143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.160238 (+ 95us) server_negotiation.cc:407] Connection header received
0807 00:40:28.160292 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.160293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.160306 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.160375 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:28.160575 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.161083 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.162145 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.162548 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.162658 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.162668 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.162852 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.162890 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:40:28.163038 13285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.159880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44075 (local address 127.1.116.129:32819)
0807 00:40:28.160135 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.160148 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.160222 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.160395 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.160396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.160413 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:28.160490 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.160491 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.161184 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.161185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.162034 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.162037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.162671 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.162672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.162758 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.162763 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.162829 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.162874 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":123,"threads_started":1}
I20260807 00:40:28.163277 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15020f8656614a64882b0ef9ba0f554d" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96" is_pre_election: true
I20260807 00:40:28.163400 12956 raft_consensus.cc:2454] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 0.
I20260807 00:40:28.163571 13155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15020f8656614a64882b0ef9ba0f554d" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688dd3db2c584419a495ac81582e0afb" is_pre_election: true
I20260807 00:40:28.163677 12662 leader_election.cc:302] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 7bb0e5e5020f4124b11df2cc73874b96; no voters:
I20260807 00:40:28.163762 13155 raft_consensus.cc:2454] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 0.
I20260807 00:40:28.163830 13283 raft_consensus.cc:2781] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:28.163915 13283 raft_consensus.cc:482] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:28.163951 13283 raft_consensus.cc:3037] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.165023 13283 raft_consensus.cc:504] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.165205 13283 leader_election.cc:288] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 election: Requested vote from peers 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313), 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075)
I20260807 00:40:28.165382 13155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15020f8656614a64882b0ef9ba0f554d" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688dd3db2c584419a495ac81582e0afb"
I20260807 00:40:28.165397 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "15020f8656614a64882b0ef9ba0f554d" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96"
I20260807 00:40:28.165472 13155 raft_consensus.cc:3037] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.165496 12956 raft_consensus.cc:3037] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.166251 13155 raft_consensus.cc:2454] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 1.
I20260807 00:40:28.166255 12956 raft_consensus.cc:2454] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 1.
I20260807 00:40:28.166482 12661 leader_election.cc:302] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 688dd3db2c584419a495ac81582e0afb; no voters:
I20260807 00:40:28.166666 13283 raft_consensus.cc:2781] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:28.166863 13283 raft_consensus.cc:686] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 1 LEADER]: Becoming Leader. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Running, Role: LEADER
I20260807 00:40:28.166980 13283 consensus_queue.cc:237] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.167976 12541 catalog_manager.cc:5495] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54674934aed84cb0a147040afdb5aea9 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } health_report { overall_health: UNKNOWN } } }
I20260807 00:40:28.182961 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.180341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54812 (local address 127.1.116.129:44769)
0807 00:40:28.180404 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.180406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.180418 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:28.180470 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.180471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.180491 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.180566 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:28.180766 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.181279 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.182177 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.182368 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.182428 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.182627 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.182755 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.182826 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.182870 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.182891 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:40:28.185286 12956 raft_consensus.cc:1261] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Refusing update from remote peer 54674934aed84cb0a147040afdb5aea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.185286 13155 raft_consensus.cc:1261] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Refusing update from remote peer 54674934aed84cb0a147040afdb5aea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.185750 13283 consensus_queue.cc:1035] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.185998 13286 consensus_queue.cc:1035] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.190178 13291 mvcc.cc:205] Tried to move back new op lower bound from 7315714982643585024 to 7315714982572822528. Current Snapshot: MvccSnapshot[applied={T|T < 7315714982643585024 or (T in {7315714982643585024})}]
I20260807 00:40:28.190364 13290 mvcc.cc:205] Tried to move back new op lower bound from 7315714982643585024 to 7315714982572822528. Current Snapshot: MvccSnapshot[applied={T|T < 7315714982643585024 or (T in {7315714982643585024})}]
I20260807 00:40:28.191748 12540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.191869 12540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.192827 12540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Sending DeleteTablet for 3 replicas of tablet 15020f8656614a64882b0ef9ba0f554d
I20260807 00:40:28.193253 13129 tablet_service.cc:1514] Processing DeleteTablet for tablet 15020f8656614a64882b0ef9ba0f554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:56638
I20260807 00:40:28.193231 12725 tablet_service.cc:1514] Processing DeleteTablet for tablet 15020f8656614a64882b0ef9ba0f554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260807 00:40:28.193480 12929 tablet_service.cc:1514] Processing DeleteTablet for tablet 15020f8656614a64882b0ef9ba0f554d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37676
I20260807 00:40:28.193722 12540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:28.193903 12540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:28.194056 13300 tablet_replica.cc:330] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: stopping tablet replica
I20260807 00:40:28.194227 13300 raft_consensus.cc:2229] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.194779 13300 raft_consensus.cc:2258] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.195461 13300 ts_tablet_manager.cc:1905] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.196156 13298 tablet_replica.cc:330] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: stopping tablet replica
I20260807 00:40:28.196334 13298 raft_consensus.cc:2229] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:28.196502 13298 raft_consensus.cc:2258] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.196724 13302 tablet_replica.cc:330] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: stopping tablet replica
I20260807 00:40:28.196877 13302 raft_consensus.cc:2229] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.197002 13298 ts_tablet_manager.cc:1905] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.197362 13302 raft_consensus.cc:2258] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.198567 13302 ts_tablet_manager.cc:1905] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.198642 12725 tablet_service.cc:1467] Processing CreateTablet for tablet e232f75cb5e54cb8baa1b0d2e0977e62 (DEFAULT_TABLE table=test-table [id=210fe38fa2524143be9eba05c6285d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.198771 12725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e232f75cb5e54cb8baa1b0d2e0977e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.199388 13298 ts_tablet_manager.cc:1918] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:28.199494 13298 log.cc:1196] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/15020f8656614a64882b0ef9ba0f554d
I20260807 00:40:28.199790 13298 ts_tablet_manager.cc:1939] T 15020f8656614a64882b0ef9ba0f554d P 54674934aed84cb0a147040afdb5aea9: Deleting consensus metadata
I20260807 00:40:28.200544 12929 tablet_service.cc:1467] Processing CreateTablet for tablet e232f75cb5e54cb8baa1b0d2e0977e62 (DEFAULT_TABLE table=test-table [id=210fe38fa2524143be9eba05c6285d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.200702 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e232f75cb5e54cb8baa1b0d2e0977e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.201009 13129 tablet_service.cc:1467] Processing CreateTablet for tablet e232f75cb5e54cb8baa1b0d2e0977e62 (DEFAULT_TABLE table=test-table [id=210fe38fa2524143be9eba05c6285d17]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.200600 12527 catalog_manager.cc:4841] TS 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769): tablet 15020f8656614a64882b0ef9ba0f554d (table test-table [id=5e9ef4e4a4f24d8da786c62fddf52784]) successfully deleted
I20260807 00:40:28.201406 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e232f75cb5e54cb8baa1b0d2e0977e62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.201622 13302 ts_tablet_manager.cc:1918] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:28.202066 13302 log.cc:1196] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/15020f8656614a64882b0ef9ba0f554d
I20260807 00:40:28.202349 13270 tablet_bootstrap.cc:492] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Bootstrap starting.
I20260807 00:40:28.202502 13302 ts_tablet_manager.cc:1939] T 15020f8656614a64882b0ef9ba0f554d P 7bb0e5e5020f4124b11df2cc73874b96: Deleting consensus metadata
I20260807 00:40:28.202896 13270 tablet_bootstrap.cc:654] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.203413 12522 catalog_manager.cc:4841] TS 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313): tablet 15020f8656614a64882b0ef9ba0f554d (table test-table [id=5e9ef4e4a4f24d8da786c62fddf52784]) successfully deleted
I20260807 00:40:28.203902 13270 tablet_bootstrap.cc:492] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: No bootstrap required, opened a new log
I20260807 00:40:28.203961 13270 ts_tablet_manager.cc:1397] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 00:40:28.204177 13269 tablet_bootstrap.cc:492] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Bootstrap starting.
I20260807 00:40:28.204155 13270 raft_consensus.cc:348] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.204237 13270 raft_consensus.cc:374] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.204258 13270 raft_consensus.cc:729] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb0e5e5020f4124b11df2cc73874b96, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.204337 13270 consensus_queue.cc:260] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.204450 13270 ts_tablet_manager.cc:1428] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:40:28.204680 13269 tablet_bootstrap.cc:654] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.205582 13269 tablet_bootstrap.cc:492] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: No bootstrap required, opened a new log
I20260807 00:40:28.205641 13269 ts_tablet_manager.cc:1397] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:28.205803 13269 raft_consensus.cc:348] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.205873 13269 raft_consensus.cc:374] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.205894 13269 raft_consensus.cc:729] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688dd3db2c584419a495ac81582e0afb, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.205951 13269 consensus_queue.cc:260] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.206030 13269 ts_tablet_manager.cc:1428] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:28.206140 13279 tablet_bootstrap.cc:492] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Bootstrap starting.
I20260807 00:40:28.207027 13279 tablet_bootstrap.cc:654] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.207304 13300 ts_tablet_manager.cc:1918] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:28.207376 13300 log.cc:1196] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/15020f8656614a64882b0ef9ba0f554d
I20260807 00:40:28.207758 13300 ts_tablet_manager.cc:1939] T 15020f8656614a64882b0ef9ba0f554d P 688dd3db2c584419a495ac81582e0afb: Deleting consensus metadata
I20260807 00:40:28.207865 13279 tablet_bootstrap.cc:492] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: No bootstrap required, opened a new log
I20260807 00:40:28.207909 13279 ts_tablet_manager.cc:1397] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:40:28.208063 13279 raft_consensus.cc:348] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.208185 13279 raft_consensus.cc:374] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.208209 13279 raft_consensus.cc:729] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.208259 13279 consensus_queue.cc:260] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.208359 13279 ts_tablet_manager.cc:1428] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:28.208654 12521 catalog_manager.cc:4841] TS 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075): tablet 15020f8656614a64882b0ef9ba0f554d (table test-table [id=5e9ef4e4a4f24d8da786c62fddf52784]) successfully deleted
W20260807 00:40:28.220757 12822 tablet.cc:2369] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:40:28.328873 13223 tablet.cc:2369] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:28.374402 13277 raft_consensus.cc:482] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:28.374591 13277 raft_consensus.cc:504] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.375174 13277 leader_election.cc:288] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769), 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313)
I20260807 00:40:28.378755 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.375612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:37247 (local address 127.1.116.129:44769)
0807 00:40:28.375691 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.375694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.375710 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:40:28.375748 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.375750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.375762 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.375951 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:28.376242 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.376740 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.377931 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.378299 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.378361 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.378368 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.378632 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.378672 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:40:28.378775 13070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.375426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44769 (local address 127.1.116.131:37247)
0807 00:40:28.375511 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.375533 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.375644 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.375978 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.375979 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.375983 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:28.376142 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.376145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.376832 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.376833 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.377821 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.377824 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.378397 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.378398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.378505 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.378510 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.378605 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.378660 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 00:40:28.378878 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.375565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:55097 (local address 127.1.116.130:33313)
0807 00:40:28.375644 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.375647 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.375802 (+ 155us) server_negotiation.cc:407] Connection header received
0807 00:40:28.375987 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.375988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.376000 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.376079 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:28.376315 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.376950 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.377975 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.378402 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.378488 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.378499 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.378743 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.378788 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:40:28.378957 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.375426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:33313 (local address 127.1.116.131:55097)
0807 00:40:28.375703 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.375716 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.375873 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.376099 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.376100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.376103 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:28.376225 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.376227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.377049 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.377050 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.377840 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.377842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.378566 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.378568 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.378648 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.378654 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.378716 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.378757 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":25,"thread_start_us":77,"threads_started":1}
I20260807 00:40:28.379196 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e232f75cb5e54cb8baa1b0d2e0977e62" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54674934aed84cb0a147040afdb5aea9" is_pre_election: true
I20260807 00:40:28.379346 12750 raft_consensus.cc:2454] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 0.
I20260807 00:40:28.379316 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e232f75cb5e54cb8baa1b0d2e0977e62" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96" is_pre_election: true
I20260807 00:40:28.379431 12956 raft_consensus.cc:2454] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 0.
I20260807 00:40:28.379640 13063 leader_election.cc:302] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 688dd3db2c584419a495ac81582e0afb; no voters:
I20260807 00:40:28.379797 13277 raft_consensus.cc:2781] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:28.379838 13277 raft_consensus.cc:482] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:28.379864 13277 raft_consensus.cc:3037] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.380846 13277 raft_consensus.cc:504] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.381040 13277 leader_election.cc:288] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 election: Requested vote from peers 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769), 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313)
I20260807 00:40:28.381283 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e232f75cb5e54cb8baa1b0d2e0977e62" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96"
I20260807 00:40:28.381307 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e232f75cb5e54cb8baa1b0d2e0977e62" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54674934aed84cb0a147040afdb5aea9"
I20260807 00:40:28.381371 12956 raft_consensus.cc:3037] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.381404 12750 raft_consensus.cc:3037] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.382136 12956 raft_consensus.cc:2454] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 1.
I20260807 00:40:28.382213 12750 raft_consensus.cc:2454] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 1.
I20260807 00:40:28.382380 13060 leader_election.cc:302] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688dd3db2c584419a495ac81582e0afb, 7bb0e5e5020f4124b11df2cc73874b96; no voters:
I20260807 00:40:28.382505 13277 raft_consensus.cc:2781] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:28.382737 13277 raft_consensus.cc:686] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 1 LEADER]: Becoming Leader. State: Replica: 688dd3db2c584419a495ac81582e0afb, State: Running, Role: LEADER
I20260807 00:40:28.382833 13277 consensus_queue.cc:237] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.383762 12540 catalog_manager.cc:5495] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb reported cstate change: term changed from 0 to 1, leader changed from <none> to 688dd3db2c584419a495ac81582e0afb (127.1.116.131). New cstate: current_term: 1 leader_uuid: "688dd3db2c584419a495ac81582e0afb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } health_report { overall_health: HEALTHY } } }
W20260807 00:40:28.396538 13028 tablet.cc:2369] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063227, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.190@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:28.424010 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.415560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50390 (local address 127.1.116.190:35543)
0807 00:40:28.415655 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.415657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.415898 (+ 241us) server_negotiation.cc:407] Connection header received
0807 00:40:28.416025 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.416026 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.416039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.416131 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:28.416707 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.417252 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.418058 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.418243 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.418343 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.421019 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:28.421021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:28.421021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:28.421031 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:28.422494 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.423084 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.423085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.423086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.423153 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.423392 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.423393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.423393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.423511 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:28.423609 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.423792 (+ 183us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.423918 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":65}
I20260807 00:40:28.424011 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.415499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35543 (local address 127.0.0.1:50390)
0807 00:40:28.415716 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.415726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.415931 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.416139 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.416140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.416561 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:28.416618 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.416621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.417376 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.417377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.417963 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.417965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.418338 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.418339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.418389 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.419059 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:28.419064 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:28.420771 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:28.422720 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:28.422729 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:28.422732 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:28.422975 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:28.423225 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:28.423226 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:28.423227 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:28.423312 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:28.423602 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:28.423610 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:28.423657 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.423741 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.423863 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":107,"threads_started":1}
I20260807 00:40:28.426445 12540 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:50390:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 00:40:28.426719 12540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:28.428689 12725 tablet_service.cc:1467] Processing CreateTablet for tablet 3017fb32b61848cbb1cb174cc00e4c0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a9739bf0f994691abd19862f36670dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:40:28.428706 12929 tablet_service.cc:1467] Processing CreateTablet for tablet 3017fb32b61848cbb1cb174cc00e4c0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a9739bf0f994691abd19862f36670dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:40:28.428915 12725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3017fb32b61848cbb1cb174cc00e4c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.428915 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3017fb32b61848cbb1cb174cc00e4c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.430783 13270 tablet_bootstrap.cc:492] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: Bootstrap starting.
I20260807 00:40:28.430805 13279 tablet_bootstrap.cc:492] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: Bootstrap starting.
I20260807 00:40:28.430764 13129 tablet_service.cc:1467] Processing CreateTablet for tablet 3017fb32b61848cbb1cb174cc00e4c0c (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1a9739bf0f994691abd19862f36670dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:40:28.430974 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3017fb32b61848cbb1cb174cc00e4c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.431233 13279 tablet_bootstrap.cc:654] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.431249 13270 tablet_bootstrap.cc:654] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.431907 13279 tablet_bootstrap.cc:492] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: No bootstrap required, opened a new log
I20260807 00:40:28.431975 13279 ts_tablet_manager.cc:1397] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:28.432037 13270 tablet_bootstrap.cc:492] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: No bootstrap required, opened a new log
I20260807 00:40:28.432088 13270 ts_tablet_manager.cc:1397] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:28.432117 13279 raft_consensus.cc:348] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.432211 13279 raft_consensus.cc:374] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.432240 13279 raft_consensus.cc:729] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.432284 13279 consensus_queue.cc:260] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.432279 13270 raft_consensus.cc:348] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.432359 13270 raft_consensus.cc:374] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.432386 13270 raft_consensus.cc:729] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb0e5e5020f4124b11df2cc73874b96, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.432397 13279 ts_tablet_manager.cc:1428] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:28.432439 13270 consensus_queue.cc:260] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.432552 13270 ts_tablet_manager.cc:1428] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:40:28.432540 13286 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.432616 13286 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:28.432596 13280 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.432685 13280 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:28.432811 13269 tablet_bootstrap.cc:492] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: Bootstrap starting.
I20260807 00:40:28.433326 13269 tablet_bootstrap.cc:654] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.434043 13269 tablet_bootstrap.cc:492] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: No bootstrap required, opened a new log
I20260807 00:40:28.434113 13269 ts_tablet_manager.cc:1397] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:28.434445 13269 raft_consensus.cc:348] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.434540 13269 raft_consensus.cc:374] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.434587 13269 raft_consensus.cc:729] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688dd3db2c584419a495ac81582e0afb, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.434657 13269 consensus_queue.cc:260] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.434785 13269 ts_tablet_manager.cc:1428] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:28.434988 13277 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.435103 13277 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:28.436733 13286 raft_consensus.cc:482] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:28.436823 13286 raft_consensus.cc:504] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.437006 13286 leader_election.cc:288] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313), 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075)
I20260807 00:40:28.437229 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3017fb32b61848cbb1cb174cc00e4c0c" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96" is_pre_election: true
I20260807 00:40:28.437304 12956 raft_consensus.cc:2454] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 0.
I20260807 00:40:28.437247 13155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3017fb32b61848cbb1cb174cc00e4c0c" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688dd3db2c584419a495ac81582e0afb" is_pre_election: true
I20260807 00:40:28.437359 13155 raft_consensus.cc:2454] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 0.
I20260807 00:40:28.437466 12662 leader_election.cc:302] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 7bb0e5e5020f4124b11df2cc73874b96; no voters:
I20260807 00:40:28.437556 13286 raft_consensus.cc:2781] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:28.437595 13286 raft_consensus.cc:482] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:28.437621 13286 raft_consensus.cc:3037] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.438344 13286 raft_consensus.cc:504] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.438486 13286 leader_election.cc:288] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 election: Requested vote from peers 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313), 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075)
I20260807 00:40:28.438648 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3017fb32b61848cbb1cb174cc00e4c0c" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96"
I20260807 00:40:28.438728 12956 raft_consensus.cc:3037] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.438719 13155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3017fb32b61848cbb1cb174cc00e4c0c" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688dd3db2c584419a495ac81582e0afb"
I20260807 00:40:28.438874 13155 raft_consensus.cc:3037] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.439401 12956 raft_consensus.cc:2454] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 1.
I20260807 00:40:28.439586 12662 leader_election.cc:302] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 7bb0e5e5020f4124b11df2cc73874b96; no voters:
I20260807 00:40:28.439702 13286 raft_consensus.cc:2781] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:28.439760 13286 raft_consensus.cc:686] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [term 1 LEADER]: Becoming Leader. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Running, Role: LEADER
I20260807 00:40:28.439819 13286 consensus_queue.cc:237] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.439947 13155 raft_consensus.cc:2454] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 1.
I20260807 00:40:28.440105 13283 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: TxnStatusTablet state changed. Reason: New leader 54674934aed84cb0a147040afdb5aea9. Latest consensus state: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.440202 13283 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:40:28.440529 13323 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 00:40:28.440676 13323 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 00:40:28.440651 12540 catalog_manager.cc:5495] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54674934aed84cb0a147040afdb5aea9 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } health_report { overall_health: UNKNOWN } } }
I20260807 00:40:28.449764 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.446365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54822 (local address 127.1.116.129:44769)
0807 00:40:28.446628 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.446630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.446641 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:40:28.446678 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.446679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.446691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.446748 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:28.446944 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.447486 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.448644 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.448922 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.449005 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.449351 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.449421 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.449487 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.449658 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.449692 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235}
I20260807 00:40:28.449995 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.446312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44769 (local address 127.0.0.1:54822)
0807 00:40:28.446433 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.446449 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.446516 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.446786 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.446788 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.446790 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:40:28.446860 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.446863 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.447573 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.447574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.448531 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.448533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.449059 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.449060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.449184 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.449211 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.449520 (+ 309us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.449531 (+ 11us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.449649 (+ 118us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.449800 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 00:40:28.451263 13155 raft_consensus.cc:1261] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Refusing update from remote peer 54674934aed84cb0a147040afdb5aea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.451498 13283 consensus_queue.cc:1035] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.451550 12956 raft_consensus.cc:1261] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Refusing update from remote peer 54674934aed84cb0a147040afdb5aea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.452082 13277 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: TxnStatusTablet state changed. Reason: New leader 54674934aed84cb0a147040afdb5aea9. Latest consensus state: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.452168 13277 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:28.452459 13283 consensus_queue.cc:1035] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.453044 13283 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.453121 13283 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:40:28.453433 13280 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: TxnStatusTablet state changed. Reason: New leader 54674934aed84cb0a147040afdb5aea9. Latest consensus state: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.453538 13280 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:28.453796 13286 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.453895 13286 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 54674934aed84cb0a147040afdb5aea9: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:40:28.454285 13280 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.454366 13280 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 7bb0e5e5020f4124b11df2cc73874b96: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:28.455116 13289 mvcc.cc:205] Tried to move back new op lower bound from 7315714983734571008 to 7315714983689777152. Current Snapshot: MvccSnapshot[applied={T|T < 7315714983734571008 or (T in {7315714983734571008})}]
I20260807 00:40:28.455847 13277 tablet_replica.cc:439] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } } }
I20260807 00:40:28.455933 13277 tablet_replica.cc:442] T 3017fb32b61848cbb1cb174cc00e4c0c P 688dd3db2c584419a495ac81582e0afb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:40:28.458443 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.455229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56652 (local address 127.1.116.131:44075)
0807 00:40:28.455465 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.455467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.455474 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:40:28.455503 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.455504 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.455522 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.455590 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:28.456052 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.456758 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.457564 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.457859 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.457947 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.458072 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.458195 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.458276 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.458360 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.458378 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216}
I20260807 00:40:28.464589 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.461820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:53237 (local address 127.1.116.129:44769)
0807 00:40:28.461908 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.461910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.461917 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:40:28.461939 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.461940 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.461950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.462002 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:28.462182 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.462714 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.463781 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.463984 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.464043 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.464286 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.464320 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.464379 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.464509 (+ 130us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.464532 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:40:28.464602 13232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.461566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44769 (local address 127.1.116.131:53237)
0807 00:40:28.461655 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.461669 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.461724 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.462036 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.462037 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.462038 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:40:28.462097 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.462098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.462800 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.462800 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.463673 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.463675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.464073 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.464074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.464149 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.464166 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.464409 (+ 243us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.464411 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.464480 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.464527 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 00:40:28.467352 12956 raft_consensus.cc:1261] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Refusing update from remote peer 688dd3db2c584419a495ac81582e0afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.467456 12750 raft_consensus.cc:1261] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Refusing update from remote peer 688dd3db2c584419a495ac81582e0afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.467685 13308 consensus_queue.cc:1035] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.467806 13277 consensus_queue.cc:1035] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.478492 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.475653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:48735 (local address 127.1.116.131:44075)
0807 00:40:28.475697 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.475699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.475878 (+ 179us) server_negotiation.cc:407] Connection header received
0807 00:40:28.475953 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.475954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.475963 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.476012 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:28.476135 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.476631 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.477729 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.477942 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.478003 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.478222 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.478255 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.478318 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.478413 (+ 95us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.478434 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:40:28.478567 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.475574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44075 (local address 127.1.116.129:48735)
0807 00:40:28.475792 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.475805 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:28.475881 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:28.476019 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:28.476019 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:28.476021 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:40:28.476067 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.476069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.476712 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.476713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.477635 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:28.477637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.478029 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.478030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:28.478086 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.478102 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.478339 (+ 237us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.478341 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:40:28.478410 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:40:28.478457 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":98,"threads_started":1}
I20260807 00:40:28.487915 12541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.488049 12541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.488791 12541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Sending DeleteTablet for 3 replicas of tablet e232f75cb5e54cb8baa1b0d2e0977e62
I20260807 00:40:28.489079 12725 tablet_service.cc:1514] Processing DeleteTablet for tablet e232f75cb5e54cb8baa1b0d2e0977e62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260807 00:40:28.489127 12929 tablet_service.cc:1514] Processing DeleteTablet for tablet e232f75cb5e54cb8baa1b0d2e0977e62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37676
I20260807 00:40:28.489185 13298 tablet_replica.cc:330] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: stopping tablet replica
I20260807 00:40:28.489252 13298 raft_consensus.cc:2229] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.489259 13302 tablet_replica.cc:330] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: stopping tablet replica
I20260807 00:40:28.489295 13298 raft_consensus.cc:2258] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.489343 13302 raft_consensus.cc:2229] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.489385 13302 raft_consensus.cc:2258] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.489471 13129 tablet_service.cc:1514] Processing DeleteTablet for tablet e232f75cb5e54cb8baa1b0d2e0977e62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:56638
I20260807 00:40:28.489570 13300 tablet_replica.cc:330] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: stopping tablet replica
I20260807 00:40:28.489630 13300 raft_consensus.cc:2229] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:28.489727 13300 raft_consensus.cc:2258] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.489770 13302 ts_tablet_manager.cc:1905] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.489984 13298 ts_tablet_manager.cc:1905] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.490923 13300 ts_tablet_manager.cc:1905] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.492373 13298 ts_tablet_manager.cc:1918] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:28.492440 13298 log.cc:1196] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/e232f75cb5e54cb8baa1b0d2e0977e62
I20260807 00:40:28.492499 13300 ts_tablet_manager.cc:1918] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:28.492569 13300 log.cc:1196] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/e232f75cb5e54cb8baa1b0d2e0977e62
I20260807 00:40:28.492735 13298 ts_tablet_manager.cc:1939] T e232f75cb5e54cb8baa1b0d2e0977e62 P 54674934aed84cb0a147040afdb5aea9: Deleting consensus metadata
I20260807 00:40:28.492861 13300 ts_tablet_manager.cc:1939] T e232f75cb5e54cb8baa1b0d2e0977e62 P 688dd3db2c584419a495ac81582e0afb: Deleting consensus metadata
I20260807 00:40:28.493189 12527 catalog_manager.cc:4841] TS 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769): tablet e232f75cb5e54cb8baa1b0d2e0977e62 (table test-table [id=210fe38fa2524143be9eba05c6285d17]) successfully deleted
I20260807 00:40:28.493201 12521 catalog_manager.cc:4841] TS 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075): tablet e232f75cb5e54cb8baa1b0d2e0977e62 (table test-table [id=210fe38fa2524143be9eba05c6285d17]) successfully deleted
I20260807 00:40:28.494050 13302 ts_tablet_manager.cc:1918] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:28.494122 13302 log.cc:1196] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/e232f75cb5e54cb8baa1b0d2e0977e62
I20260807 00:40:28.494342 13302 ts_tablet_manager.cc:1939] T e232f75cb5e54cb8baa1b0d2e0977e62 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting consensus metadata
I20260807 00:40:28.494722 12522 catalog_manager.cc:4841] TS 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313): tablet e232f75cb5e54cb8baa1b0d2e0977e62 (table test-table [id=210fe38fa2524143be9eba05c6285d17]) successfully deleted
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:40:28.535356 12611 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:28.520114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50396 (local address 127.1.116.190:35543)
0807 00:40:28.520166 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.520168 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.520474 (+ 306us) server_negotiation.cc:407] Connection header received
0807 00:40:28.520591 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.520592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.520604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.520634 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:28.521411 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.522257 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.523528 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.523837 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.523881 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.535287 (+ 11406us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50396: BlockingRecv error: recv got EOF from 127.0.0.1:50396 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[12479](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063228, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM
I20260807 00:40:28.546499 12611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.536757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50410 (local address 127.1.116.190:35543)
0807 00:40:28.536793 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.536795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.536924 (+ 129us) server_negotiation.cc:407] Connection header received
0807 00:40:28.537052 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.537053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.537069 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.537103 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:28.537793 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.538613 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.539726 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.540043 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.540125 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.543283 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:28.543284 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:28.543285 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:28.543292 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:28.545075 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.545539 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.545540 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.545541 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.545606 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:28.545889 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:28.545890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:28.545891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:28.546039 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:28.546162 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.546330 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.546416 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:40:28.548934 12541 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:28.549129 12541 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:28.550900 12929 tablet_service.cc:1467] Processing CreateTablet for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 (DEFAULT_TABLE table=test-table [id=195badaa46274fa58d69c583d9bb4162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.551054 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.551057 13129 tablet_service.cc:1467] Processing CreateTablet for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 (DEFAULT_TABLE table=test-table [id=195badaa46274fa58d69c583d9bb4162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.551227 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.552484 13270 tablet_bootstrap.cc:492] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: Bootstrap starting.
I20260807 00:40:28.552830 13269 tablet_bootstrap.cc:492] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: Bootstrap starting.
I20260807 00:40:28.552953 13270 tablet_bootstrap.cc:654] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.553304 13269 tablet_bootstrap.cc:654] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.553422 12725 tablet_service.cc:1467] Processing CreateTablet for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 (DEFAULT_TABLE table=test-table [id=195badaa46274fa58d69c583d9bb4162]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.553565 12725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.554956 13270 tablet_bootstrap.cc:492] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: No bootstrap required, opened a new log
I20260807 00:40:28.555018 13270 ts_tablet_manager.cc:1397] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 00:40:28.555060 13279 tablet_bootstrap.cc:492] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: Bootstrap starting.
I20260807 00:40:28.555205 13270 raft_consensus.cc:348] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.555286 13270 raft_consensus.cc:374] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.555307 13270 raft_consensus.cc:729] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb0e5e5020f4124b11df2cc73874b96, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.555341 13270 consensus_queue.cc:260] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.555356 13279 tablet_bootstrap.cc:654] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.555434 13270 ts_tablet_manager.cc:1428] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:28.556574 13269 tablet_bootstrap.cc:492] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: No bootstrap required, opened a new log
I20260807 00:40:28.556650 13269 ts_tablet_manager.cc:1397] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260807 00:40:28.556833 13269 raft_consensus.cc:348] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.556929 13269 raft_consensus.cc:374] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.556953 13269 raft_consensus.cc:729] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688dd3db2c584419a495ac81582e0afb, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.557011 13269 consensus_queue.cc:260] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.557137 13269 ts_tablet_manager.cc:1428] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:40:28.558983 13279 tablet_bootstrap.cc:492] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: No bootstrap required, opened a new log
I20260807 00:40:28.559046 13279 ts_tablet_manager.cc:1397] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 00:40:28.559154 13279 raft_consensus.cc:348] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.559227 13279 raft_consensus.cc:374] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.559248 13279 raft_consensus.cc:729] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.559291 13279 consensus_queue.cc:260] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.559381 13279 ts_tablet_manager.cc:1428] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260807 00:40:28.606957 13333 raft_consensus.cc:482] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:28.607115 13333 raft_consensus.cc:504] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.607345 13333 leader_election.cc:288] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313), 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075)
I20260807 00:40:28.607585 13155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4251e0ce3f44fc5a0fadc8ee6b99b67" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688dd3db2c584419a495ac81582e0afb" is_pre_election: true
I20260807 00:40:28.607605 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4251e0ce3f44fc5a0fadc8ee6b99b67" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96" is_pre_election: true
I20260807 00:40:28.607729 12956 raft_consensus.cc:2454] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 0.
I20260807 00:40:28.607748 13155 raft_consensus.cc:2454] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 0.
I20260807 00:40:28.607937 12661 leader_election.cc:302] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 688dd3db2c584419a495ac81582e0afb; no voters:
I20260807 00:40:28.608053 13333 raft_consensus.cc:2781] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:28.608124 13333 raft_consensus.cc:482] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:28.608144 13333 raft_consensus.cc:3037] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.609323 13333 raft_consensus.cc:504] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.609478 13333 leader_election.cc:288] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 election: Requested vote from peers 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313), 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075)
I20260807 00:40:28.609799 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4251e0ce3f44fc5a0fadc8ee6b99b67" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96"
I20260807 00:40:28.609817 13155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4251e0ce3f44fc5a0fadc8ee6b99b67" candidate_uuid: "54674934aed84cb0a147040afdb5aea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "688dd3db2c584419a495ac81582e0afb"
I20260807 00:40:28.609898 12956 raft_consensus.cc:3037] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.609912 13155 raft_consensus.cc:3037] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.610725 13155 raft_consensus.cc:2454] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 1.
I20260807 00:40:28.610877 12956 raft_consensus.cc:2454] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 54674934aed84cb0a147040afdb5aea9 in term 1.
I20260807 00:40:28.611070 12661 leader_election.cc:302] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 688dd3db2c584419a495ac81582e0afb; no voters:
I20260807 00:40:28.611245 13333 raft_consensus.cc:2781] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:28.611320 13333 raft_consensus.cc:686] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 1 LEADER]: Becoming Leader. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Running, Role: LEADER
I20260807 00:40:28.611404 13333 consensus_queue.cc:237] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.612134 12541 catalog_manager.cc:5495] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 54674934aed84cb0a147040afdb5aea9 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "54674934aed84cb0a147040afdb5aea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } health_report { overall_health: UNKNOWN } } }
I20260807 00:40:28.619127 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.616409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.1.116.129:44769)
0807 00:40:28.616474 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.616476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.616487 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:40:28.616562 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.616562 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.616571 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.616624 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:28.616796 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.617241 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.618440 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.618699 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.618760 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.618937 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.618966 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.619011 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.619067 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.619081 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:40:28.620306 12956 raft_consensus.cc:1261] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Refusing update from remote peer 54674934aed84cb0a147040afdb5aea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.620311 13155 raft_consensus.cc:1261] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Refusing update from remote peer 54674934aed84cb0a147040afdb5aea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.620684 13333 consensus_queue.cc:1035] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.620841 13286 consensus_queue.cc:1035] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.623804 12542 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.623914 12542 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.624883 12542 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Sending DeleteTablet for 3 replicas of tablet e4251e0ce3f44fc5a0fadc8ee6b99b67
I20260807 00:40:28.625171 12725 tablet_service.cc:1514] Processing DeleteTablet for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260807 00:40:28.625270 13298 tablet_replica.cc:330] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: stopping tablet replica
I20260807 00:40:28.625252 12929 tablet_service.cc:1514] Processing DeleteTablet for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37676
I20260807 00:40:28.625330 13298 raft_consensus.cc:2229] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:28.625351 13302 tablet_replica.cc:330] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: stopping tablet replica
I20260807 00:40:28.625396 13298 raft_consensus.cc:2258] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.625415 13302 raft_consensus.cc:2229] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.625475 13302 raft_consensus.cc:2258] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.625686 13298 ts_tablet_manager.cc:1905] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.625881 12542 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:28.626055 12542 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:28.626387 13129 tablet_service.cc:1514] Processing DeleteTablet for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:56638
I20260807 00:40:28.626488 13300 tablet_replica.cc:330] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: stopping tablet replica
I20260807 00:40:28.626579 13300 raft_consensus.cc:2229] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.626633 13300 raft_consensus.cc:2258] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.627135 13300 ts_tablet_manager.cc:1905] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.627300 13298 ts_tablet_manager.cc:1918] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:28.627348 13298 log.cc:1196] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/e4251e0ce3f44fc5a0fadc8ee6b99b67
I20260807 00:40:28.627552 13298 ts_tablet_manager.cc:1939] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 54674934aed84cb0a147040afdb5aea9: Deleting consensus metadata
I20260807 00:40:28.628101 13302 ts_tablet_manager.cc:1905] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.628530 13300 ts_tablet_manager.cc:1918] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:28.628571 13300 log.cc:1196] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/e4251e0ce3f44fc5a0fadc8ee6b99b67
I20260807 00:40:28.628720 13300 ts_tablet_manager.cc:1939] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 688dd3db2c584419a495ac81582e0afb: Deleting consensus metadata
I20260807 00:40:28.629278 12527 catalog_manager.cc:4841] TS 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769): tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 (table test-table [id=195badaa46274fa58d69c583d9bb4162]) successfully deleted
I20260807 00:40:28.629382 12521 catalog_manager.cc:4841] TS 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075): tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 (table test-table [id=195badaa46274fa58d69c583d9bb4162]) successfully deleted
I20260807 00:40:28.629639 13302 ts_tablet_manager.cc:1918] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:28.629717 13302 log.cc:1196] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/e4251e0ce3f44fc5a0fadc8ee6b99b67
I20260807 00:40:28.629933 13302 ts_tablet_manager.cc:1939] T e4251e0ce3f44fc5a0fadc8ee6b99b67 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting consensus metadata
I20260807 00:40:28.630271 12522 catalog_manager.cc:4841] TS 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313): tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 (table test-table [id=195badaa46274fa58d69c583d9bb4162]) successfully deleted
I20260807 00:40:28.630586 12929 tablet_service.cc:1467] Processing CreateTablet for tablet 26118af0b10c4b3bb17fe6cec4d08a70 (DEFAULT_TABLE table=test-table [id=2a90b829d40f4e0c8812cac3a8c164e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.630610 13129 tablet_service.cc:1467] Processing CreateTablet for tablet 26118af0b10c4b3bb17fe6cec4d08a70 (DEFAULT_TABLE table=test-table [id=2a90b829d40f4e0c8812cac3a8c164e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.630708 13129 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26118af0b10c4b3bb17fe6cec4d08a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.630707 12929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26118af0b10c4b3bb17fe6cec4d08a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.632076 13269 tablet_bootstrap.cc:492] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: Bootstrap starting.
I20260807 00:40:28.632597 13269 tablet_bootstrap.cc:654] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.633255 13270 tablet_bootstrap.cc:492] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: Bootstrap starting.
I20260807 00:40:28.633278 12725 tablet_service.cc:1467] Processing CreateTablet for tablet 26118af0b10c4b3bb17fe6cec4d08a70 (DEFAULT_TABLE table=test-table [id=2a90b829d40f4e0c8812cac3a8c164e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:28.633373 13269 tablet_bootstrap.cc:492] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: No bootstrap required, opened a new log
I20260807 00:40:28.633380 12725 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 26118af0b10c4b3bb17fe6cec4d08a70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:28.633435 13269 ts_tablet_manager.cc:1397] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:28.633661 13270 tablet_bootstrap.cc:654] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.633615 13269 raft_consensus.cc:348] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.633713 13269 raft_consensus.cc:374] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.633735 13269 raft_consensus.cc:729] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688dd3db2c584419a495ac81582e0afb, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.633798 13269 consensus_queue.cc:260] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.633890 13269 ts_tablet_manager.cc:1428] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:40:28.634181 12929 tablet_service.cc:1514] Processing DeleteTablet for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37676
I20260807 00:40:28.634565 13270 tablet_bootstrap.cc:492] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: No bootstrap required, opened a new log
I20260807 00:40:28.634608 13270 ts_tablet_manager.cc:1397] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:28.634763 13270 raft_consensus.cc:348] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
W20260807 00:40:28.634796 12522 catalog_manager.cc:4808] TS 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313): delete failed for tablet e4251e0ce3f44fc5a0fadc8ee6b99b67 because the tablet was not found. No further retry: Not found: Tablet not found: e4251e0ce3f44fc5a0fadc8ee6b99b67
I20260807 00:40:28.634842 13270 raft_consensus.cc:374] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.634843 13279 tablet_bootstrap.cc:492] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9: Bootstrap starting.
I20260807 00:40:28.634871 13270 raft_consensus.cc:729] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bb0e5e5020f4124b11df2cc73874b96, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.634943 13270 consensus_queue.cc:260] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.635041 13270 ts_tablet_manager.cc:1428] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:28.635248 13279 tablet_bootstrap.cc:654] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:28.635758 13279 tablet_bootstrap.cc:492] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9: No bootstrap required, opened a new log
I20260807 00:40:28.635810 13279 ts_tablet_manager.cc:1397] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:28.635910 13279 raft_consensus.cc:348] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.635983 13279 raft_consensus.cc:374] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:28.636010 13279 raft_consensus.cc:729] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54674934aed84cb0a147040afdb5aea9, State: Initialized, Role: FOLLOWER
I20260807 00:40:28.636053 13279 consensus_queue.cc:260] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.636154 13279 ts_tablet_manager.cc:1428] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:28.646426 13338 raft_consensus.cc:482] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:28.646522 13338 raft_consensus.cc:504] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.646718 13338 leader_election.cc:288] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769), 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313)
I20260807 00:40:28.646919 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26118af0b10c4b3bb17fe6cec4d08a70" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54674934aed84cb0a147040afdb5aea9" is_pre_election: true
I20260807 00:40:28.646944 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26118af0b10c4b3bb17fe6cec4d08a70" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96" is_pre_election: true
I20260807 00:40:28.647004 12750 raft_consensus.cc:2454] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 0.
I20260807 00:40:28.647035 12956 raft_consensus.cc:2454] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 0.
I20260807 00:40:28.647235 13060 leader_election.cc:302] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 688dd3db2c584419a495ac81582e0afb, 7bb0e5e5020f4124b11df2cc73874b96; no voters:
I20260807 00:40:28.647369 13338 raft_consensus.cc:2781] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:28.647410 13338 raft_consensus.cc:482] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:28.647437 13338 raft_consensus.cc:3037] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.648139 13338 raft_consensus.cc:504] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.648278 13338 leader_election.cc:288] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 election: Requested vote from peers 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769), 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313)
I20260807 00:40:28.648484 12750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26118af0b10c4b3bb17fe6cec4d08a70" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "54674934aed84cb0a147040afdb5aea9"
I20260807 00:40:28.648545 12750 raft_consensus.cc:3037] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.649469 12750 raft_consensus.cc:2454] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 1.
I20260807 00:40:28.649607 12956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "26118af0b10c4b3bb17fe6cec4d08a70" candidate_uuid: "688dd3db2c584419a495ac81582e0afb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7bb0e5e5020f4124b11df2cc73874b96"
I20260807 00:40:28.649734 12956 raft_consensus.cc:3037] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:28.649719 13063 leader_election.cc:302] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 54674934aed84cb0a147040afdb5aea9, 688dd3db2c584419a495ac81582e0afb; no voters:
I20260807 00:40:28.650619 12956 raft_consensus.cc:2454] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 688dd3db2c584419a495ac81582e0afb in term 1.
I20260807 00:40:28.650652 13338 raft_consensus.cc:2781] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:28.650738 13338 raft_consensus.cc:686] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 1 LEADER]: Becoming Leader. State: Replica: 688dd3db2c584419a495ac81582e0afb, State: Running, Role: LEADER
I20260807 00:40:28.650805 13338 consensus_queue.cc:237] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } }
I20260807 00:40:28.651567 12542 catalog_manager.cc:5495] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb reported cstate change: term changed from 0 to 1, leader changed from <none> to 688dd3db2c584419a495ac81582e0afb (127.1.116.131). New cstate: current_term: 1 leader_uuid: "688dd3db2c584419a495ac81582e0afb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "688dd3db2c584419a495ac81582e0afb" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44075 } health_report { overall_health: HEALTHY } } }
I20260807 00:40:28.659337 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.656869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56654 (local address 127.1.116.131:44075)
0807 00:40:28.656931 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:28.656934 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:28.656965 (+ 31us) server_negotiation.cc:407] Connection header received
0807 00:40:28.657047 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:28.657048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:28.657065 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:28.657109 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:28.657275 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.657809 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.658623 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:28.658827 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:28.658905 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:28.659059 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:28.659100 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:28.659162 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:28.659225 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 00:40:28.659247 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 00:40:28.662016 12956 raft_consensus.cc:1261] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Refusing update from remote peer 688dd3db2c584419a495ac81582e0afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.662014 12750 raft_consensus.cc:1261] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Refusing update from remote peer 688dd3db2c584419a495ac81582e0afb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:28.662264 13338 consensus_queue.cc:1035] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7bb0e5e5020f4124b11df2cc73874b96" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.662380 13308 consensus_queue.cc:1035] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [LEADER]: Connected to new peer: Peer: permanent_uuid: "54674934aed84cb0a147040afdb5aea9" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:28.676522 12541 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.676643 12541 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:28.677419 12541 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c9318c034b6a4d34bbb8fa9651720e2f: Sending DeleteTablet for 3 replicas of tablet 26118af0b10c4b3bb17fe6cec4d08a70
I20260807 00:40:28.677704 12725 tablet_service.cc:1514] Processing DeleteTablet for tablet 26118af0b10c4b3bb17fe6cec4d08a70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:54808
I20260807 00:40:28.677718 13129 tablet_service.cc:1514] Processing DeleteTablet for tablet 26118af0b10c4b3bb17fe6cec4d08a70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:56638
I20260807 00:40:28.677862 13298 tablet_replica.cc:330] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9: stopping tablet replica
I20260807 00:40:28.677881 13300 tablet_replica.cc:330] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: stopping tablet replica
I20260807 00:40:28.677940 13298 raft_consensus.cc:2229] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.677953 13300 raft_consensus.cc:2229] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:40:28.677984 13298 raft_consensus.cc:2258] T 26118af0b10c4b3bb17fe6cec4d08a70 P 54674934aed84cb0a147040afdb5aea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.678011 13300 raft_consensus.cc:2258] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.678174 12929 tablet_service.cc:1514] Processing DeleteTablet for tablet 26118af0b10c4b3bb17fe6cec4d08a70 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:28 UTC) from {username='oryx', principal='oryx/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37676
I20260807 00:40:28.678201 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 12640
I20260807 00:40:28.678262 13302 tablet_replica.cc:330] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: stopping tablet replica
I20260807 00:40:28.678308 13302 raft_consensus.cc:2229] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:28.678342 13302 raft_consensus.cc:2258] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:28.681921 13302 ts_tablet_manager.cc:1905] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.681934 13300 ts_tablet_manager.cc:1905] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:28.683440 13300 ts_tablet_manager.cc:1918] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:28.683483 13300 log.cc:1196] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/26118af0b10c4b3bb17fe6cec4d08a70
I20260807 00:40:28.683671 13300 ts_tablet_manager.cc:1939] T 26118af0b10c4b3bb17fe6cec4d08a70 P 688dd3db2c584419a495ac81582e0afb: Deleting consensus metadata
I20260807 00:40:28.683741 13302 ts_tablet_manager.cc:1918] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:40:28.683801 13302 log.cc:1196] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/26118af0b10c4b3bb17fe6cec4d08a70
I20260807 00:40:28.684070 13302 ts_tablet_manager.cc:1939] T 26118af0b10c4b3bb17fe6cec4d08a70 P 7bb0e5e5020f4124b11df2cc73874b96: Deleting consensus metadata
I20260807 00:40:28.684437 12522 catalog_manager.cc:4841] TS 7bb0e5e5020f4124b11df2cc73874b96 (127.1.116.130:33313): tablet 26118af0b10c4b3bb17fe6cec4d08a70 (table test-table [id=2a90b829d40f4e0c8812cac3a8c164e5]) successfully deleted
I20260807 00:40:28.686803 12521 catalog_manager.cc:4841] TS 688dd3db2c584419a495ac81582e0afb (127.1.116.131:44075): tablet 26118af0b10c4b3bb17fe6cec4d08a70 (table test-table [id=2a90b829d40f4e0c8812cac3a8c164e5]) successfully deleted
W20260807 00:40:28.694943 12527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:44769 (error 108)
I20260807 00:40:28.695217 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 12833
I20260807 00:40:28.700003 13378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:28.699740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44769 (local address 127.0.0.1:54848)
0807 00:40:28.699888 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:28.699950 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:44769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
W20260807 00:40:28.701180 12527 catalog_manager.cc:4584] TS 54674934aed84cb0a147040afdb5aea9 (127.1.116.129:44769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 26118af0b10c4b3bb17fe6cec4d08a70: Network error: Client connection negotiation failed: client connection to 127.1.116.129:44769: connect: Connection refused (error 111)
I20260807 00:40:28.705974 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 13039
I20260807 00:40:28.721976 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 12499
2026-08-07T00:40:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2731 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[13384](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[13384](info): set up 2 sockets
Aug 07 00:40:28 dist-test-slave-h6n0 krb5kdc[13384](info): commencing operation
krb5kdc: starting...
W20260807 00:40:29.988457 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:30 dist-test-slave-h6n0 krb5kdc[13384](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063230, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:30.121742 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36999
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:30.216174 13403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:30.216424 13403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:30.216449 13403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:30.217679 13403 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:30.217726 13403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:30.217743 13403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:30.217757 13403 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:30.217821 13403 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:30.219448 13403 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:30.219744 13403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:30.220026 13403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:30.225654 13415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:30.225669 13413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:30.225659 13412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:30.226115 13403 server_base.cc:1034] running on GCE node
I20260807 00:40:30.226431 13403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:30.226871 13403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:30.228053 13403 hybrid_clock.cc:648] HybridClock initialized: now 1786063230228034 us; error 61 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T00:40:30Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1513 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:30 dist-test-slave-h6n0 krb5kdc[13425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:30 dist-test-slave-h6n0 krb5kdc[13425](info): set up 2 sockets
Aug 07 00:40:30 dist-test-slave-h6n0 krb5kdc[13425](info): commencing operation
krb5kdc: starting...
W20260807 00:40:31.481441 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:31 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:31.619338 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38399
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:38399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:31.711165 13442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:31.711391 13442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:31.711414 13442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:31.712512 13442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:31.712549 13442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:31.712564 13442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:31.712577 13442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:31.712589 13442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:31.714015 13442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:38399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:31.714242 13442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:31.714445 13442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:31.719909 13450 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:31.719911 13452 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:31.719915 13449 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:31.720261 13442 server_base.cc:1034] running on GCE node
I20260807 00:40:31.720588 13442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:31.721007 13442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:31.722213 13442 hybrid_clock.cc:648] HybridClock initialized: now 1786063231722204 us; error 55 us; skew 500 ppm
Aug 07 00:40:31 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:31.725260 13442 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:31.725775 13442 webserver.cc:460] Webserver started at http://127.1.116.190:44777/ using document root <none> and password file <none>
I20260807 00:40:31.726032 13442 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:31.726087 13442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:31.726356 13442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:31.727607 13442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "edf9ea808ee0427aaaa8bed2765de215"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "e2215995897d7d9c5b1749a6a6efee91"
server_key_iv: "9bcc1d6be6e8867a44c7721f7c3a9d1f"
server_key_version: "encryptionkey@0"
I20260807 00:40:31.728031 13442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "edf9ea808ee0427aaaa8bed2765de215"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "e2215995897d7d9c5b1749a6a6efee91"
server_key_iv: "9bcc1d6be6e8867a44c7721f7c3a9d1f"
server_key_version: "encryptionkey@0"
I20260807 00:40:31.729589 13442 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:31.730504 13461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:31.730813 13442 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:31.730942 13442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "edf9ea808ee0427aaaa8bed2765de215"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "e2215995897d7d9c5b1749a6a6efee91"
server_key_iv: "9bcc1d6be6e8867a44c7721f7c3a9d1f"
server_key_version: "encryptionkey@0"
I20260807 00:40:31.731026 13442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:31.750058 13442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:31.752563 13442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:31.752722 13442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:31.757751 13442 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:38399
I20260807 00:40:31.758147 13442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:31.758217 13540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:38399 every 8 connection(s)
I20260807 00:40:31.759081 13541 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:31.761837 13541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Bootstrap starting.
I20260807 00:40:31.762632 13541 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:31.762969 13541 log.cc:824] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:31.763826 13541 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: No bootstrap required, opened a new log
I20260807 00:40:31.765631 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 13442
I20260807 00:40:31.765769 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:31.766000 1490 external_mini_cluster.cc:1428] Setting key c80b73bfa35757b6713d638c8cc5c4bb
I20260807 00:40:31.765900 13541 raft_consensus.cc:348] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:31.766093 13541 raft_consensus.cc:374] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:31.766144 13541 raft_consensus.cc:729] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf9ea808ee0427aaaa8bed2765de215, State: Initialized, Role: FOLLOWER
I20260807 00:40:31.766414 13541 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:31.766619 13541 raft_consensus.cc:388] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:31.766705 13541 raft_consensus.cc:482] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:31.766778 13541 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:31.767474 13541 raft_consensus.cc:504] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:31.767580 13541 leader_election.cc:302] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edf9ea808ee0427aaaa8bed2765de215; no voters:
I20260807 00:40:31.767815 13541 leader_election.cc:288] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:31.767889 13547 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:31.768106 13547 raft_consensus.cc:686] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 LEADER]: Becoming Leader. State: Replica: edf9ea808ee0427aaaa8bed2765de215, State: Running, Role: LEADER
I20260807 00:40:31.768214 13541 sys_catalog.cc:564] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:31.768355 13547 consensus_queue.cc:237] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:31.769836 13548 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edf9ea808ee0427aaaa8bed2765de215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } } }
I20260807 00:40:31.770005 13549 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edf9ea808ee0427aaaa8bed2765de215. Latest consensus state: current_term: 1 leader_uuid: "edf9ea808ee0427aaaa8bed2765de215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } } }
I20260807 00:40:31.770248 13548 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:31.770262 13549 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: This master's current role is: LEADER
W20260807 00:40:31.770553 13571 catalog_manager.cc:1558] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:40:31.770612 13571 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:31.770712 13573 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:31.770925 13573 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:31.773124 13573 catalog_manager.cc:1347] Generated new cluster ID: e640a942d5f943c1a81dd264c16ccce5
I20260807 00:40:31.773190 13573 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:40:31 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063231, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:31.778908 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:31.766754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47200 (local address 127.1.116.190:38399)
0807 00:40:31.766989 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:31.766997 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:31.767024 (+ 27us) server_negotiation.cc:407] Connection header received
0807 00:40:31.767435 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:31.767442 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:31.767489 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:31.767654 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:31.768550 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:31.770223 (+ 1673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:31.771457 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:31.771985 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:31.772123 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:31.774934 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:31.774939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:31.774939 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:31.774951 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:31.776971 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:31.777328 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:31.777331 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:31.777331 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:31.777448 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:31.777625 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:31.777626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:31.777627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:31.777923 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:31.778077 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:31.778509 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 00:40:31.778683 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":167,"thread_start_us":118,"threads_started":1}
I20260807 00:40:31.782240 13573 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:31.783016 13573 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:31.786788 13573 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Generated new TSK 0
I20260807 00:40:31.787022 13573 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:40:31.821862 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38399
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:31.908526 13581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:31.908735 13581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:31.908758 13581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:31.909893 13581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:31.909937 13581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:31.909963 13581 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:31.911403 13581 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:31.911634 13581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:31.911882 13581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:31.917285 13588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:31.917299 13591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:31.917305 13587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:31.918067 13581 server_base.cc:1034] running on GCE node
I20260807 00:40:31.918295 13581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:31.918571 13581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:31.919744 13581 hybrid_clock.cc:648] HybridClock initialized: now 1786063231919728 us; error 35 us; skew 500 ppm
Aug 07 00:40:31 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:31.922300 13581 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:40:31.922755 13581 webserver.cc:460] Webserver started at http://127.1.116.129:38191/ using document root <none> and password file <none>
I20260807 00:40:31.922935 13581 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:31.922978 13581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:31.923074 13581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:31.923996 13581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "4d16c76e1b494de7a3b13dfdf01e8795"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "d520466bdac73e1e2154a3f34bc49f15"
server_key_iv: "d98cce0bbba73b30b97484b150e9826c"
server_key_version: "encryptionkey@0"
I20260807 00:40:31.924293 13581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "4d16c76e1b494de7a3b13dfdf01e8795"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "d520466bdac73e1e2154a3f34bc49f15"
server_key_iv: "d98cce0bbba73b30b97484b150e9826c"
server_key_version: "encryptionkey@0"
I20260807 00:40:31.925567 13581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:31.926326 13603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:31.926606 13581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:31.926712 13581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "4d16c76e1b494de7a3b13dfdf01e8795"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "d520466bdac73e1e2154a3f34bc49f15"
server_key_iv: "d98cce0bbba73b30b97484b150e9826c"
server_key_version: "encryptionkey@0"
I20260807 00:40:31.926795 13581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:31.935772 13581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:31.937734 13581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:31.937878 13581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:31.938097 13581 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:31.938391 13581 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:31.938446 13581 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:31.938479 13581 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:31.938505 13581 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:31.945309 13581 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:44589
I20260807 00:40:31.945360 13757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:44589 every 8 connection(s)
I20260807 00:40:31.945647 13581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:31.947561 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 13581
I20260807 00:40:31.947700 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:31.947952 1490 external_mini_cluster.cc:1428] Setting key ff0a6c41f0ed14340b7e89d961eeb53f
Aug 07 00:40:31 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063231, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:31.956669 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:31.946473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:59613 (local address 127.1.116.190:38399)
0807 00:40:31.946570 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:31.946573 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:31.946851 (+ 278us) server_negotiation.cc:407] Connection header received
0807 00:40:31.947328 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:31.947330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:31.947343 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:31.947418 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:31.948589 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:31.949284 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:31.950403 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:31.950701 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:31.950786 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:31.953249 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:31.953251 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:31.953252 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:31.953268 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:31.955084 (+ 1816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:31.955531 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:31.955533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:31.955533 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:31.955602 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:31.955872 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:31.955873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:31.955873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:31.956048 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:31.956131 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:31.956449 (+ 318us) server_negotiation.cc:299] Negotiation successful
0807 00:40:31.956554 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:40:31.956709 13760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:31.946433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:59613)
0807 00:40:31.946709 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:31.946746 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:31.947188 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:31.947530 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:31.947535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:31.948305 (+ 770us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:31.948483 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:31.948487 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:31.949413 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:31.949414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:31.950297 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:31.950301 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:31.950802 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:31.950803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:31.950941 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:31.951552 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:31.951561 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:31.953126 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:31.955185 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:31.955193 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:31.955195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:31.955434 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:31.955685 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:31.955686 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:31.955686 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:31.955792 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:31.956156 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:31.956163 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:31.956297 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 00:40:31.956391 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 00:40:31.956524 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":195,"threads_started":1}
I20260807 00:40:31.957329 13758 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38399
I20260807 00:40:31.957438 13758 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:40:31.957655 13758 heartbeater.cc:511] Master 127.1.116.190:38399 requested a full tablet report, sending...
I20260807 00:40:31.958292 13488 ts_manager.cc:194] Registered new tserver with Master: 4d16c76e1b494de7a3b13dfdf01e8795 (127.1.116.129:44589)
I20260807 00:40:31.959045 13488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:59613
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:40:31.975049 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38399
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:32.055236 13763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:32.055460 13763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:32.055500 13763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:32.056612 13763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:32.056679 13763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:32.056725 13763 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:32.058162 13763 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:32.058419 13763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:32.058714 13763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:32.064213 13770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.064203 13771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.064505 13763 server_base.cc:1034] running on GCE node
W20260807 00:40:32.064186 13779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.064826 13763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:32.065140 13763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:32.066337 13763 hybrid_clock.cc:648] HybridClock initialized: now 1786063232066315 us; error 36 us; skew 500 ppm
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:32.069777 13763 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:40:32.070221 13763 webserver.cc:460] Webserver started at http://127.1.116.130:40385/ using document root <none> and password file <none>
I20260807 00:40:32.070452 13763 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:32.070498 13763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:32.070672 13763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:32.071774 13763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "00f58522f4d449d88ed9cf6b63a5a8df"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "cc439f15611a7664ac74312a475f0112"
server_key_iv: "a1b8750fd7253923383a118a543a932a"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.072086 13763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "00f58522f4d449d88ed9cf6b63a5a8df"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "cc439f15611a7664ac74312a475f0112"
server_key_iv: "a1b8750fd7253923383a118a543a932a"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.073426 13763 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:32.074255 13792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.074469 13763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:32.074599 13763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "00f58522f4d449d88ed9cf6b63a5a8df"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "cc439f15611a7664ac74312a475f0112"
server_key_iv: "a1b8750fd7253923383a118a543a932a"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.074709 13763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:32.086741 13763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:32.089103 13763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:32.089282 13763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:32.089527 13763 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:32.089872 13763 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:32.089927 13763 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.089959 13763 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:32.089990 13763 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.097496 13763 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:34981
I20260807 00:40:32.097546 13953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:34981 every 8 connection(s)
I20260807 00:40:32.097930 13763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:32.100947 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 13763
I20260807 00:40:32.101082 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:32.101301 1490 external_mini_cluster.cc:1428] Setting key e669b53f4b305c4e865e1b006d752b38
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:32.108982 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.098825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:33469 (local address 127.1.116.190:38399)
0807 00:40:32.098904 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.098906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.099108 (+ 202us) server_negotiation.cc:407] Connection header received
0807 00:40:32.099686 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.099688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.099700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.099774 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.100775 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.101409 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.102982 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.103172 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.103234 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.105893 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:32.105895 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:32.105896 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:32.105904 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:32.107511 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:32.107944 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:32.107945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:32.107946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:32.108009 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:32.108281 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:32.108282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:32.108283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:32.108411 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:32.108487 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:32.108777 (+ 290us) server_negotiation.cc:299] Negotiation successful
0807 00:40:32.108882 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:40:32.109200 13957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.098782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:33469)
0807 00:40:32.098959 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.099005 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.099548 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.099832 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.099835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.100434 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.100647 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.100651 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.101526 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.101527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.102875 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.102879 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.103269 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.103270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.103434 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.104082 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.104094 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.105776 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:32.107605 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:32.107613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:32.107615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:32.107848 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:32.108089 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:32.108090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:32.108091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:32.108209 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:32.108488 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:32.108496 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:32.108639 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 00:40:32.108755 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 00:40:32.108926 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260807 00:40:32.109710 13954 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38399
I20260807 00:40:32.109812 13954 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:32.110030 13954 heartbeater.cc:511] Master 127.1.116.190:38399 requested a full tablet report, sending...
I20260807 00:40:32.110486 13488 ts_manager.cc:194] Registered new tserver with Master: 00f58522f4d449d88ed9cf6b63a5a8df (127.1.116.130:34981)
I20260807 00:40:32.111085 13488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:33469
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:40:32.130950 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38399
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:32.215720 13960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:32.215909 13960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:32.215930 13960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:32.216972 13960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:32.217010 13960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:32.217036 13960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:32.218340 13960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13960
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:32.218509 13960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:32.218760 13960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:32.224076 13970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.224129 13966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.224095 13965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.224458 13960 server_base.cc:1034] running on GCE node
I20260807 00:40:32.224709 13960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:32.224944 13960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:32.226112 13960 hybrid_clock.cc:648] HybridClock initialized: now 1786063232226092 us; error 34 us; skew 500 ppm
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:32.228863 13960 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:40:32.229327 13960 webserver.cc:460] Webserver started at http://127.1.116.131:33441/ using document root <none> and password file <none>
I20260807 00:40:32.229604 13960 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:32.229673 13960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:32.229811 13960 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:32.230870 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "c511ab33191a4eb89b1358d186e3d1be"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "ea035a1d7408c98bca038c808cbc065e"
server_key_iv: "f6ab1f013c480f267f5d7a0c376efa4d"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.231288 13960 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "c511ab33191a4eb89b1358d186e3d1be"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "ea035a1d7408c98bca038c808cbc065e"
server_key_iv: "f6ab1f013c480f267f5d7a0c376efa4d"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.232636 13960 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 00:40:32.233578 13982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.233829 13960 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:32.233949 13960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "c511ab33191a4eb89b1358d186e3d1be"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "ea035a1d7408c98bca038c808cbc065e"
server_key_iv: "f6ab1f013c480f267f5d7a0c376efa4d"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.234053 13960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:32.260761 13960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:32.262809 13960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:32.263063 13960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:32.263295 13960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:32.263674 13960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:32.263720 13960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.263754 13960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:32.263782 13960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.271010 13960 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:45683
I20260807 00:40:32.271181 14145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:45683 every 8 connection(s)
I20260807 00:40:32.271410 13960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:32.277704 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 13960
I20260807 00:40:32.277858 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:32.278254 1490 external_mini_cluster.cc:1428] Setting key c02970375e22e3a1e029a6aaa6962c74
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:32.283406 13545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.272145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:42237 (local address 127.1.116.190:38399)
0807 00:40:32.272243 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.272246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.272450 (+ 204us) server_negotiation.cc:407] Connection header received
0807 00:40:32.272947 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.272948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.272961 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.273023 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.273928 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.274431 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.276134 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.276375 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.276456 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.279800 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:32.279802 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:32.279802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:32.279809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:32.281762 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:32.282270 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:32.282271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:32.282272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:32.282378 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:32.282721 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:32.282722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:32.282722 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:32.282888 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:32.282993 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:32.283253 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 00:40:32.283332 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260807 00:40:32.283497 14148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.272124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:42237)
0807 00:40:32.272307 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.272346 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.272840 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.273094 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.273098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.273657 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.273831 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.273835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.274553 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.274555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.276017 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.276020 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.276472 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.276473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.276603 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.277262 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.277294 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.279690 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:32.281865 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:32.281873 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:32.281875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:32.282166 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:32.282465 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:32.282466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:32.282467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:32.282635 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:32.282992 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:32.283003 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:32.283118 (+ 115us) client_negotiation.cc:769] Sending connection context
0807 00:40:32.283201 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 00:40:32.283313 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260807 00:40:32.284060 14146 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38399
I20260807 00:40:32.284168 14146 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:32.284341 14146 heartbeater.cc:511] Master 127.1.116.190:38399 requested a full tablet report, sending...
I20260807 00:40:32.284871 13488 ts_manager.cc:194] Registered new tserver with Master: c511ab33191a4eb89b1358d186e3d1be (127.1.116.131:45683)
I20260807 00:40:32.285287 13488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:42237
I20260807 00:40:32.290982 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab.
Entry for principal oryx/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab.
WARNING: no policy specified for oryx/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab.
Entry for principal oryx/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab.
WARNING: no policy specified for oryx/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab.
Entry for principal oryx/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab.
I20260807 00:40:32.372787 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 13581
I20260807 00:40:32.385180 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 13763
I20260807 00:40:32.397123 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 13960
I20260807 00:40:32.408321 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 13442
I20260807 00:40:32.421267 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38399
--webserver_interface=127.1.116.190
--webserver_port=44777
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:38399
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:32.514086 14161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:32.514360 14161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:32.514387 14161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:32.516415 14161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:32.516471 14161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:32.516487 14161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:32.516502 14161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:32.516518 14161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:32.518240 14161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:38399
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=44777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:32.518574 14161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:32.518879 14161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:32.524708 14174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.524685 14171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.524730 14170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.525095 14161 server_base.cc:1034] running on GCE node
I20260807 00:40:32.525394 14161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:32.525823 14161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:32.526994 14161 hybrid_clock.cc:648] HybridClock initialized: now 1786063232526976 us; error 71 us; skew 500 ppm
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:32.534226 14161 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:32.534734 14161 webserver.cc:460] Webserver started at http://127.1.116.190:44777/ using document root <none> and password file <none>
I20260807 00:40:32.534942 14161 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:32.534996 14161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:32.536456 14161 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:32.537262 14187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.537542 14161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:32.537665 14161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "edf9ea808ee0427aaaa8bed2765de215"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "e2215995897d7d9c5b1749a6a6efee91"
server_key_iv: "9bcc1d6be6e8867a44c7721f7c3a9d1f"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.538058 14161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:32.548727 14161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:32.550814 14161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:32.550966 14161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:32.555328 14161 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:38399
I20260807 00:40:32.555420 14257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:38399 every 8 connection(s)
I20260807 00:40:32.555778 14161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:32.557190 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 14161
I20260807 00:40:32.557575 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:44589
--local_ip_for_outbound_sockets=127.1.116.129
--tserver_master_addrs=127.1.116.190:38399
--webserver_port=38191
--webserver_interface=127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:40:32.557904 14258 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:40:32.559173 14258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Bootstrap starting.
I20260807 00:40:32.570293 14258 log.cc:824] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:32.572271 14258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:40:32.572598 14258 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Bootstrap complete.
I20260807 00:40:32.575258 14258 raft_consensus.cc:348] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:32.575501 14258 raft_consensus.cc:729] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf9ea808ee0427aaaa8bed2765de215, State: Initialized, Role: FOLLOWER
I20260807 00:40:32.575695 14258 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:32.575778 14258 raft_consensus.cc:388] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:32.575834 14258 raft_consensus.cc:482] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:32.575888 14258 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:40:32.577014 14258 raft_consensus.cc:504] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:32.577149 14258 leader_election.cc:302] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edf9ea808ee0427aaaa8bed2765de215; no voters:
I20260807 00:40:32.577399 14258 leader_election.cc:288] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:40:32.577467 14265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:40:32.577684 14265 raft_consensus.cc:686] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [term 2 LEADER]: Becoming Leader. State: Replica: edf9ea808ee0427aaaa8bed2765de215, State: Running, Role: LEADER
I20260807 00:40:32.577721 14258 sys_catalog.cc:564] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:32.579365 14265 consensus_queue.cc:237] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } }
I20260807 00:40:32.579964 14266 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edf9ea808ee0427aaaa8bed2765de215. Latest consensus state: current_term: 2 leader_uuid: "edf9ea808ee0427aaaa8bed2765de215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } } }
I20260807 00:40:32.580111 14266 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:32.580173 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "edf9ea808ee0427aaaa8bed2765de215" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edf9ea808ee0427aaaa8bed2765de215" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38399 } } }
I20260807 00:40:32.580247 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:32.580365 14289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:32.580991 14289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:32.581197 14289 catalog_manager.cc:1259] Loaded cluster ID: e640a942d5f943c1a81dd264c16ccce5
I20260807 00:40:32.581266 14289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:32.582481 14289 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:32.582727 14289 catalog_manager.cc:5879] T 00000000000000000000000000000000 P edf9ea808ee0427aaaa8bed2765de215: Loaded TSK: 0
I20260807 00:40:32.583022 14289 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:40:32.646319 14261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:32.646510 14261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:32.646584 14261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:32.648267 14261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:32.648322 14261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:32.648358 14261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:32.649777 14261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.129.keytab
--principal=oryx/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:44589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=38191
--tserver_master_addrs=127.1.116.190:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:32.649963 14261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:32.650290 14261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:32.655326 14307 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.655416 14309 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.655652 14305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.655995 14261 server_base.cc:1034] running on GCE node
I20260807 00:40:32.656189 14261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:32.656454 14261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:32.657624 14261 hybrid_clock.cc:648] HybridClock initialized: now 1786063232657590 us; error 56 us; skew 500 ppm
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:32.660205 14261 init.cc:377] Logged in from keytab as oryx/127.1.116.129@KRBTEST.COM (short username oryx)
I20260807 00:40:32.660717 14261 webserver.cc:460] Webserver started at http://127.1.116.129:38191/ using document root <none> and password file <none>
I20260807 00:40:32.660945 14261 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:32.661031 14261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:32.662353 14261 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:32.663132 14318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.663407 14261 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:40:32.663522 14261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "4d16c76e1b494de7a3b13dfdf01e8795"
format_stamp: "Formatted at 2026-08-07 00:40:31 on dist-test-slave-h6n0"
server_key: "d520466bdac73e1e2154a3f34bc49f15"
server_key_iv: "d98cce0bbba73b30b97484b150e9826c"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.663838 14261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:32.676343 14261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:32.678797 14261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:32.678906 14261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:32.679210 14261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:32.679538 14261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:32.679606 14261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.679641 14261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:32.679669 14261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.686441 14261 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:44589
I20260807 00:40:32.686704 14475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:44589 every 8 connection(s)
I20260807 00:40:32.686967 14261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:32.694720 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 14261
I20260807 00:40:32.695065 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:34981
--local_ip_for_outbound_sockets=127.1.116.130
--tserver_master_addrs=127.1.116.190:38399
--webserver_port=40385
--webserver_interface=127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.697912 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.687768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:43747)
0807 00:40:32.687996 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.688052 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.688807 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.689548 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.689553 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.690121 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.690305 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.690310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.692084 (+ 1774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.692085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.692888 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.692890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.693735 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.693736 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.693836 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.694454 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.694483 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.697681 (+ 3198us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
W20260807 00:40:32.698046 14479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.687849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:43747 (local address 127.1.116.190:38399)
0807 00:40:32.689046 (+ 1197us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.689055 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.689090 (+ 35us) server_negotiation.cc:407] Connection header received
0807 00:40:32.689240 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.689246 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.689298 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.689463 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.690460 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.691958 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.692999 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.693627 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.693753 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.697897 (+ 4144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:43747: BlockingRecv error: recv got EOF from 127.1.116.129:43747 (error 108)
Metrics: {"server-negotiator.queue_time_us":1140,"thread_start_us":89,"threads_started":1}
W20260807 00:40:32.698581 14476 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:38399 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.116.190:38399: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.704921 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.698870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:41669)
0807 00:40:32.699156 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.699169 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.699217 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.699429 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.699430 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.699597 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.699664 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.699666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.700757 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.700758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.701581 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.701583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.702160 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.702161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.702235 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.702860 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.702866 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.704851 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265}
W20260807 00:40:32.705322 14481 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.698990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:41669 (local address 127.1.116.190:38399)
0807 00:40:32.699199 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.699202 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.699214 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:32.699303 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.699304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.699320 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.699408 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.699752 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.700647 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.701687 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.702060 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.702146 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.705217 (+ 3071us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:41669: BlockingRecv error: recv got EOF from 127.1.116.129:41669 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":113,"threads_started":1}
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.710767 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.705174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:32847)
0807 00:40:32.705488 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.705499 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.705539 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.705746 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.705747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.705888 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.705920 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.705921 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.706926 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.706928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.707704 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.707706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.708236 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.708237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.708294 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.708746 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.708751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.710704 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":91,"threads_started":1}
W20260807 00:40:32.710886 14476 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:40:32.711036 14481 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.705362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:32847 (local address 127.1.116.190:38399)
0807 00:40:32.705604 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.705607 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.705619 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:32.705643 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.705644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.705656 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.705728 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.706014 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.706819 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.707803 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.708134 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.708211 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.710803 (+ 2592us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:32847: BlockingRecv error: recv got EOF from 127.1.116.129:32847 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":87,"threads_started":1}
W20260807 00:40:32.780340 14480 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:32.780541 14480 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:32.780565 14480 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:32.781826 14480 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:32.781872 14480 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:32.781898 14480 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:32.783366 14480 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.130.keytab
--principal=oryx/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:34981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=40385
--tserver_master_addrs=127.1.116.190:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14480
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:32.783620 14480 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:32.783865 14480 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:32.790477 14491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.790464 14490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.791184 14480 server_base.cc:1034] running on GCE node
W20260807 00:40:32.790493 14495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.792126 14480 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:32.792459 14480 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:32.793670 14480 hybrid_clock.cc:648] HybridClock initialized: now 1786063232793644 us; error 47 us; skew 500 ppm
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:32.797331 14480 init.cc:377] Logged in from keytab as oryx/127.1.116.130@KRBTEST.COM (short username oryx)
I20260807 00:40:32.797930 14480 webserver.cc:460] Webserver started at http://127.1.116.130:40385/ using document root <none> and password file <none>
I20260807 00:40:32.798264 14480 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:32.798374 14480 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:32.800532 14480 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:32.801403 14505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.801725 14480 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:32.801846 14480 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "00f58522f4d449d88ed9cf6b63a5a8df"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "cc439f15611a7664ac74312a475f0112"
server_key_iv: "a1b8750fd7253923383a118a543a932a"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.802386 14480 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:32.817606 14480 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:32.820142 14480 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:32.820267 14480 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:32.820506 14480 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:32.820931 14480 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:32.820982 14480 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.821014 14480 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:32.821031 14480 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.828649 14480 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:34981
I20260807 00:40:32.828698 14666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:34981 every 8 connection(s)
I20260807 00:40:32.829095 14480 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:32.831573 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 14480
I20260807 00:40:32.831914 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:45683
--local_ip_for_outbound_sockets=127.1.116.131
--tserver_master_addrs=127.1.116.190:38399
--webserver_port=33441
--webserver_interface=127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:40:32.837843 14481 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.829944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:48465 (local address 127.1.116.190:38399)
0807 00:40:32.830044 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.830046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.830278 (+ 232us) server_negotiation.cc:407] Connection header received
0807 00:40:32.830941 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.830942 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.830959 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.831025 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.832264 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.833068 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.834249 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.834445 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.834506 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.837786 (+ 3280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:48465: BlockingRecv error: recv got EOF from 127.1.116.130:48465 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:40:32.837886 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.829897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:48465)
0807 00:40:32.830112 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.830159 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.830823 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.831110 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.831114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.831853 (+ 739us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.832137 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.832144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.833192 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.833193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.834150 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.834153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.834580 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.834581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.834735 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.835392 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.835403 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.837679 (+ 2276us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":114,"threads_started":1}
W20260807 00:40:32.838179 14667 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:38399 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.116.190:38399: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.843008 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.838412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:58687)
0807 00:40:32.838465 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.838475 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.838518 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.838777 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.838778 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.838911 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.838951 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.838953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.839611 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.839612 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.840140 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.840141 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.840494 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.840495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.840577 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.841038 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.841043 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.842968 (+ 1925us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 00:40:32.843253 14481 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.838509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:58687 (local address 127.1.116.190:38399)
0807 00:40:32.838638 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.838642 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.838655 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:32.838678 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.838678 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.838689 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.838759 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.839028 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.839520 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.840216 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.840410 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.840493 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.843156 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:58687: BlockingRecv error: recv got EOF from 127.1.116.130:58687 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.847970 14669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.843276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:40835)
0807 00:40:32.843368 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.843379 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.843414 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.843585 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.843586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.843696 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.843720 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.843720 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.844739 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.844739 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.845207 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.845208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.845741 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.845742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.845776 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.846172 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.846177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.847929 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260807 00:40:32.848131 14479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.843370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:40835 (local address 127.1.116.190:38399)
0807 00:40:32.843426 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.843429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.843462 (+ 33us) server_negotiation.cc:407] Connection header received
0807 00:40:32.843488 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.843488 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.843506 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.843575 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.843807 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.844636 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.845297 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.845646 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.845727 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.848051 (+ 2324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:40835: BlockingRecv error: recv got EOF from 127.1.116.130:40835 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260807 00:40:32.848178 14667 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:40:32.926117 14670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:32.926337 14670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:32.926362 14670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:32.927600 14670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:32.927646 14670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:32.927673 14670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:32.929163 14670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/krb5kdc/oryx_127.1.116.131.keytab
--principal=oryx/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:45683
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=33441
--tserver_master_addrs=127.1.116.190:38399
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:32.929430 14670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:32.929692 14670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:32.935216 14676 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.935205 14677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:32.935176 14680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:32.935515 14670 server_base.cc:1034] running on GCE node
I20260807 00:40:32.935761 14670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:32.936084 14670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:32.937271 14670 hybrid_clock.cc:648] HybridClock initialized: now 1786063232937255 us; error 36 us; skew 500 ppm
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063232, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:32.940335 14670 init.cc:377] Logged in from keytab as oryx/127.1.116.131@KRBTEST.COM (short username oryx)
I20260807 00:40:32.940877 14670 webserver.cc:460] Webserver started at http://127.1.116.131:33441/ using document root <none> and password file <none>
I20260807 00:40:32.941130 14670 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:32.941217 14670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:32.942837 14670 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:32.943498 14693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.943720 14670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:32.943836 14670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "c511ab33191a4eb89b1358d186e3d1be"
format_stamp: "Formatted at 2026-08-07 00:40:32 on dist-test-slave-h6n0"
server_key: "ea035a1d7408c98bca038c808cbc065e"
server_key_iv: "f6ab1f013c480f267f5d7a0c376efa4d"
server_key_version: "encryptionkey@0"
I20260807 00:40:32.944235 14670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:32.965873 14670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:32.968324 14670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:32.968489 14670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:32.968729 14670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:32.969058 14670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:32.969117 14670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.969153 14670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:32.969179 14670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:32.976014 14670 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:45683
I20260807 00:40:32.976155 14851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:45683 every 8 connection(s)
I20260807 00:40:32.976457 14670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:32.979655 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 14670
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.985129 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.977326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:40645)
0807 00:40:32.977538 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.977595 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.978343 (+ 748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.978694 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.978701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.979499 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.979849 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.979855 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.980752 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.980754 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.981575 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.981578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.982016 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.982017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.982147 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.982680 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.982688 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.984919 (+ 2231us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
W20260807 00:40:32.985164 14479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.977385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:40645 (local address 127.1.116.190:38399)
0807 00:40:32.977498 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.977501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.977712 (+ 211us) server_negotiation.cc:407] Connection header received
0807 00:40:32.978470 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.978472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.978494 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.978595 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.979995 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.980633 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.981676 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.981915 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.982006 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.985033 (+ 3027us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:40645: BlockingRecv error: recv got EOF from 127.1.116.131:40645 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 00:40:32.985605 14852 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:38399 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.116.190:38399: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
I20260807 00:40:32.988193 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.980334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47208 (local address 127.1.116.190:38399)
0807 00:40:32.980379 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.980381 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.980529 (+ 148us) server_negotiation.cc:407] Connection header received
0807 00:40:32.980621 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.980622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.980632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.980692 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.981349 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.981976 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.982857 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.983066 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.983157 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.984541 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:32.984546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:32.984547 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:32.984560 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:32.986640 (+ 2080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:32.987038 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:32.987040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:32.987041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:32.987125 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:32.987381 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:32.987382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:32.987382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:32.987607 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:32.987752 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:32.987933 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 00:40:32.988054 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.990787 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.985895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:35763)
0807 00:40:32.985946 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.985956 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.985992 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.986125 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.986125 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.986266 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.986317 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.986318 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.987200 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.987201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.987720 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.987721 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.988101 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.988102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.988174 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.988809 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.988814 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.990701 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 00:40:32.990844 14479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.985964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:35763 (local address 127.1.116.190:38399)
0807 00:40:32.986018 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.986020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.986029 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:32.986050 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.986050 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.986063 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.986122 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.986403 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.987118 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.987814 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.988015 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.988103 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.990783 (+ 2680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:35763: BlockingRecv error: recv got EOF from 127.1.116.131:35763 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:32 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:32.995368 14854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:32.991150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:56373)
0807 00:40:32.991313 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:32.991324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:32.991398 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:32.991526 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:32.991526 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:32.991655 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:32.991682 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.991683 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.992292 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.992293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.992810 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:32.992812 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.993179 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.993180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:32.993231 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.993583 (+ 352us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:32.993587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:32.995289 (+ 1702us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147}
W20260807 00:40:32.995415 14479 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:32.991227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:56373 (local address 127.1.116.190:38399)
0807 00:40:32.991272 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:32.991275 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:32.991384 (+ 109us) server_negotiation.cc:407] Connection header received
0807 00:40:32.991455 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:32.991456 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:32.991466 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:32.991521 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:32.991767 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.992211 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.992894 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:32.993090 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:32.993167 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:32.995355 (+ 2188us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:56373: BlockingRecv error: recv got EOF from 127.1.116.131:56373 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 00:40:32.995538 14852 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 00:40:33 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:33 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:33.716840 14857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:33.711346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:39187)
0807 00:40:33.711558 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:33.711573 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:33.711662 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:33.711831 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:33.711832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:33.712050 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:33.712156 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:33.712158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.712830 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.712831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:33.713461 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:33.713462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.713921 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.713922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:33.713980 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:33.714744 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:33.714751 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:33.716724 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":119,"threads_started":1}
W20260807 00:40:33.716881 14858 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:33.711504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:39187 (local address 127.1.116.190:38399)
0807 00:40:33.711649 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:33.711651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:33.711664 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:33.711726 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:33.711727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:33.711738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:33.711810 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:33.712238 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.712753 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.713611 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.713814 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.713884 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:33.716806 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:39187: BlockingRecv error: recv got EOF from 127.1.116.129:39187 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
Aug 07 00:40:33 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:33 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:33.854017 14859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:33.848532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:57465)
0807 00:40:33.848666 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:33.848676 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:33.848717 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:33.848910 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:33.848911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:33.849083 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:33.849137 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:33.849139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.849976 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.849978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:33.850800 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:33.850804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.851208 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.851209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:33.851262 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:33.851806 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:33.851811 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:33.853952 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":54,"threads_started":1}
W20260807 00:40:33.854210 14858 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:33.848657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:57465 (local address 127.1.116.190:38399)
0807 00:40:33.848735 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:33.848737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:33.848747 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:33.848776 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:33.848777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:33.848791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:33.848862 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:33.849234 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.849847 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.850894 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.851103 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.851174 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:33.854106 (+ 2932us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:57465: BlockingRecv error: recv got EOF from 127.1.116.130:57465 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 00:40:33 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:33 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:34.001749 14860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:33.995998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:54763)
0807 00:40:33.996166 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:33.996177 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:33.996230 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:33.996533 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:33.996535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:33.996785 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:33.996858 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:33.996860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.997702 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.997703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:33.998258 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:33.998259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.998716 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.998717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:33.998780 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:33.999323 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:33.999328 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:34.001650 (+ 2322us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":97,"threads_started":1}
W20260807 00:40:34.001823 14858 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:33.996244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:54763 (local address 127.1.116.190:38399)
0807 00:40:33.996329 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:33.996332 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:33.996344 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:33.996380 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:33.996380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:33.996397 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:33.996489 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:33.996959 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.997633 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.998351 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:33.998624 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:33.998706 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:34.001763 (+ 3057us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:54763: BlockingRecv error: recv got EOF from 127.1.116.131:54763 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Aug 07 00:40:34 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:34 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:34.723026 14861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:34.717511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:46037)
0807 00:40:34.717684 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:34.717694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:34.717746 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:34.718021 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:34.718022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:34.718194 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:34.718253 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:34.718255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.719043 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.719044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:34.719600 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:34.719602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.719956 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.719957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:34.720002 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:34.720535 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:34.720539 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:34.722934 (+ 2395us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":108,"threads_started":1}
W20260807 00:40:34.723342 14863 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:34.717667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:46037 (local address 127.1.116.190:38399)
0807 00:40:34.717853 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:34.717855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:34.717864 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:34.717892 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:34.717893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:34.717905 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:34.717985 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:34.718357 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.718956 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.719684 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.719876 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.719932 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:34.723271 (+ 3339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:46037: BlockingRecv error: recv got EOF from 127.1.116.129:46037 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":115,"threads_started":1}
Aug 07 00:40:34 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:34 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:34.860698 14864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:34.854660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:49253)
0807 00:40:34.854844 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:34.854855 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:34.854914 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:34.855146 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:34.855147 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:34.855343 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:34.855401 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:34.855403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.856496 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.856498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:34.857180 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:34.857183 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.857634 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.857635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:34.857690 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:34.858255 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:34.858261 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:34.860623 (+ 2362us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":99,"threads_started":1}
W20260807 00:40:34.860906 14863 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:34.854831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:49253 (local address 127.1.116.190:38399)
0807 00:40:34.854924 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:34.854927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:34.854968 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:40:34.855001 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:34.855002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:34.855016 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:34.855068 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:34.855517 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.856374 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.857326 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:34.857537 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:34.857634 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:34.860821 (+ 3187us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:49253: BlockingRecv error: recv got EOF from 127.1.116.130:49253 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 00:40:35 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:35 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:35.007745 14865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:35.002375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:51483)
0807 00:40:35.002610 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:35.002650 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:35.002716 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:35.002885 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:35.002886 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:35.003075 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:35.003124 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:35.003126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.003827 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.003828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:35.004352 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:35.004353 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.004725 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.004725 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:35.004780 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:35.005368 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:35.005373 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:35.007671 (+ 2298us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
W20260807 00:40:35.007854 14863 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:35.002604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51483 (local address 127.1.116.190:38399)
0807 00:40:35.002694 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:35.002696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:35.002707 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:40:35.002782 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:35.002783 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:35.002799 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:35.002877 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:35.003218 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.003729 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.004442 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.004644 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.004726 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:35.007755 (+ 3029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:51483: BlockingRecv error: recv got EOF from 127.1.116.131:51483 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 00:40:35 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:35 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:35.729219 14867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:35.723596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:39171)
0807 00:40:35.723802 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:35.723814 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:35.723918 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:35.724160 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:35.724161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:35.724386 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:35.724460 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:35.724462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.725342 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.725343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:35.726079 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:35.726082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.726444 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.726444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:35.726490 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:35.727124 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:35.727130 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:35.729142 (+ 2012us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":127,"threads_started":1}
W20260807 00:40:35.729378 14868 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:35.723730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:39171 (local address 127.1.116.190:38399)
0807 00:40:35.723891 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:35.723893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:35.723911 (+ 18us) server_negotiation.cc:407] Connection header received
0807 00:40:35.724033 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:35.724035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:35.724049 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:35.724125 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:35.724550 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.725231 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.726174 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.726363 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.726423 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:35.729256 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:39171: BlockingRecv error: recv got EOF from 127.1.116.129:39171 (error 108)
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":125,"threads_started":1}
Aug 07 00:40:35 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:35 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:35.867059 14869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:35.861310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:38621)
0807 00:40:35.861490 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:35.861502 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:35.861576 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:35.861779 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:35.861780 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:35.861985 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:35.862060 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:35.862062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.862828 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.862829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:35.863445 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:35.863446 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.863807 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.863807 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:35.863858 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:35.864353 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:35.864358 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:35.866957 (+ 2599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
W20260807 00:40:35.867223 14868 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:35.861435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:38621 (local address 127.1.116.190:38399)
0807 00:40:35.861498 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:35.861500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:35.861582 (+ 82us) server_negotiation.cc:407] Connection header received
0807 00:40:35.861622 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:35.861622 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:35.861633 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:35.861703 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:35.862180 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.862750 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.863536 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:35.863721 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:35.863783 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:35.867111 (+ 3328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:38621: BlockingRecv error: recv got EOF from 127.1.116.130:38621 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Aug 07 00:40:36 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:36 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:36.013607 14871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:36.008330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:57333)
0807 00:40:36.008519 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:36.008535 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:36.008599 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:36.008788 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:36.008789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:36.008965 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:36.009020 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:36.009022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.009776 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.009777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:36.010444 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:36.010447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.010923 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.010924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:36.010969 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:36.011512 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:36.011517 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:36.013538 (+ 2021us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":126,"threads_started":1}
W20260807 00:40:36.013866 14868 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:36.008435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:57333 (local address 127.1.116.190:38399)
0807 00:40:36.008544 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:36.008547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:36.008651 (+ 104us) server_negotiation.cc:407] Connection header received
0807 00:40:36.008680 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:36.008681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:36.008699 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:36.008749 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:36.009107 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.009672 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.010605 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.010832 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.010897 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:36.013667 (+ 2770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:57333: BlockingRecv error: recv got EOF from 127.1.116.131:57333 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 00:40:36 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:36 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:36.735129 14872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:36.729794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:60097)
0807 00:40:36.730010 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:36.730024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:36.730085 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:36.730234 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:36.730235 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:36.730430 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:36.730489 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:36.730491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.731289 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.731290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:36.731864 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:36.731865 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.732221 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.732222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:36.732279 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:36.732846 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:36.732850 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:36.735012 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":120,"threads_started":1}
W20260807 00:40:36.735183 14873 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:36.729923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:60097 (local address 127.1.116.190:38399)
0807 00:40:36.730084 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:36.730087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:36.730097 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:36.730133 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:36.730134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:36.730148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:36.730227 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:36.730623 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.731201 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.731961 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.732147 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.732203 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:36.735102 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:60097: BlockingRecv error: recv got EOF from 127.1.116.129:60097 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":86,"threads_started":1}
Aug 07 00:40:36 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:36 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:36.873173 14874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:36.867636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:57919)
0807 00:40:36.867829 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:36.867842 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:36.867903 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:36.868179 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:36.868180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:36.868362 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:36.868419 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:36.868421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.869325 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.869325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:36.869875 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:36.869876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.870269 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.870269 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:36.870310 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:36.870875 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:36.870880 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:36.873057 (+ 2177us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":100,"threads_started":1}
W20260807 00:40:36.873258 14873 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:36.867795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:57919 (local address 127.1.116.190:38399)
0807 00:40:36.867892 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:36.867895 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:36.867909 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:40:36.867984 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:36.867985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:36.868001 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:36.868062 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:36.868514 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.869244 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.869964 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:36.870187 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:36.870251 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:36.873186 (+ 2935us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:57919: BlockingRecv error: recv got EOF from 127.1.116.130:57919 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 07 00:40:37 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:37 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:37.019767 14875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:37.014232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:54829)
0807 00:40:37.014434 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:37.014455 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:37.014557 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:37.014732 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:37.014734 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:37.014948 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:37.015020 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:37.015021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.015743 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.015744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:37.016328 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:37.016330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.016723 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.016724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:37.016768 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:37.017402 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:37.017408 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:37.019656 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":97,"threads_started":1}
W20260807 00:40:37.019804 14873 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:37.014314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:54829 (local address 127.1.116.190:38399)
0807 00:40:37.014441 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:37.014443 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:37.014544 (+ 101us) server_negotiation.cc:407] Connection header received
0807 00:40:37.014616 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:37.014617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:37.014631 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:37.014701 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:37.015120 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.015650 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.016435 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.016635 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.016705 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:37.019739 (+ 3034us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:54829: BlockingRecv error: recv got EOF from 127.1.116.131:54829 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 07 00:40:37 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:37 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:37.745438 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:37.735756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:56233)
0807 00:40:37.735886 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:37.735897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:37.735945 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:37.736270 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:37.736271 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:37.736431 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:37.736488 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:37.736490 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.737257 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.737258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:37.737798 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:37.737800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.738184 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.738185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:37.738239 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:37.743311 (+ 5072us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:37.743315 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:37.745374 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":69,"threads_started":1}
W20260807 00:40:37.745707 14877 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:37.735869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:56233 (local address 127.1.116.190:38399)
0807 00:40:37.736082 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:37.736084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:37.736098 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:40:37.736136 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:37.736136 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:37.736150 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:37.736234 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:37.736579 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.737165 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.737896 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.738098 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.738157 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:37.745596 (+ 7439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:56233: BlockingRecv error: recv got EOF from 127.1.116.129:56233 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":139,"threads_started":1}
Aug 07 00:40:37 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:37 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:37.884449 14878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:37.873942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:60343)
0807 00:40:37.874099 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:37.874113 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:37.874185 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:37.874415 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:37.874416 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:37.874702 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:37.874765 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:37.874767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.875528 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.875529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:37.876087 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:37.876088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.876466 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.876467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:37.876509 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:37.881838 (+ 5329us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:37.881844 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:37.884385 (+ 2541us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
W20260807 00:40:37.884691 14877 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:37.874085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:60343 (local address 127.1.116.190:38399)
0807 00:40:37.874160 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:37.874162 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:37.874237 (+ 75us) server_negotiation.cc:407] Connection header received
0807 00:40:37.874275 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:37.874276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:37.874297 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:37.874350 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:37.874865 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.875420 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.876187 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:37.876382 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:37.876446 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:37.884548 (+ 8102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:60343: BlockingRecv error: recv got EOF from 127.1.116.130:60343 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 07 00:40:38 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:38 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:38.025301 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:38.020391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:46215)
0807 00:40:38.020632 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:38.020646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:38.020745 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:38.020922 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:38.020923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:38.021120 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:38.021170 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:38.021171 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.021865 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.021866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:38.022390 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:38.022392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.022838 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.022839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:38.022900 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:38.023437 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:38.023442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:38.025206 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":123,"threads_started":1}
W20260807 00:40:38.025339 14877 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:38.020508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:46215 (local address 127.1.116.190:38399)
0807 00:40:38.020598 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:38.020601 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:38.020721 (+ 120us) server_negotiation.cc:407] Connection header received
0807 00:40:38.020810 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:38.020811 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:38.020827 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:38.020898 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:38.021267 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.021769 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.022485 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.022747 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.022829 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:38.025278 (+ 2449us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:46215: BlockingRecv error: recv got EOF from 127.1.116.131:46215 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 00:40:38 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:38 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:38.751205 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:38.746084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:41031)
0807 00:40:38.746254 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:38.746266 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:38.746328 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:38.746563 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:38.746564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:38.746801 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:38.746859 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:38.746861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.747744 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.747745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:38.748345 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:38.748347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.748761 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.748762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:38.748813 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:38.749360 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:38.749365 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:38.751144 (+ 1779us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
W20260807 00:40:38.751394 14883 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:38.746179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:41031 (local address 127.1.116.190:38399)
0807 00:40:38.746354 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:38.746356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:38.746365 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:40:38.746412 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:38.746413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:38.746427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:38.746511 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:38.746980 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.747592 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.748447 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.748655 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.748718 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:38.751267 (+ 2549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:41031: BlockingRecv error: recv got EOF from 127.1.116.129:41031 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
Aug 07 00:40:38 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:38 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:38.892689 14885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:38.885174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:35195)
0807 00:40:38.887177 (+ 2003us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:38.887193 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:38.887316 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:38.887516 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:38.887517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:38.887722 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:38.887768 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:38.887770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.888550 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.888551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:38.889219 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:38.889222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.889667 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.889668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:38.889723 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:38.890277 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:38.890282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:38.892611 (+ 2329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1974,"thread_start_us":1769,"threads_started":1}
W20260807 00:40:38.892886 14883 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:38.887015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:35195 (local address 127.1.116.190:38399)
0807 00:40:38.887092 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:38.887095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:38.887327 (+ 232us) server_negotiation.cc:407] Connection header received
0807 00:40:38.887422 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:38.887423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:38.887437 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:38.887495 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:38.887859 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.888459 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.889325 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:38.889582 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:38.889657 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:38.892786 (+ 3129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:35195: BlockingRecv error: recv got EOF from 127.1.116.130:35195 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 07 00:40:39 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:39 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:39.031337 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:39.025943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:48255)
0807 00:40:39.026186 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:39.026200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:39.026276 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:39.026447 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:39.026448 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:39.026655 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:39.026711 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:39.026712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.027476 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.027476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:39.028061 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:39.028063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.028451 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.028452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:39.028495 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:39.029027 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:39.029031 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:39.031265 (+ 2234us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":122,"threads_started":1}
W20260807 00:40:39.031515 14883 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:39.026065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:48255 (local address 127.1.116.190:38399)
0807 00:40:39.026201 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:39.026204 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:39.026278 (+ 74us) server_negotiation.cc:407] Connection header received
0807 00:40:39.026332 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:39.026333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:39.026348 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:39.026408 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:39.026801 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.027389 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.028190 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.028373 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.028433 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:39.031425 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:48255: BlockingRecv error: recv got EOF from 127.1.116.131:48255 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 07 00:40:39 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:39 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:39.757191 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:39.751909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:52183)
0807 00:40:39.752173 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:39.752188 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:39.752250 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:39.752417 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:39.752418 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:39.752599 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:39.752655 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:39.752657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.753405 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.753406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:39.753979 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:39.753981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.754379 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.754381 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:39.754434 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:39.754986 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:39.754991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:39.757119 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":122,"threads_started":1}
W20260807 00:40:39.757395 14888 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:39.752021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:52183 (local address 127.1.116.190:38399)
0807 00:40:39.752236 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:39.752239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:39.752259 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:40:39.752297 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:39.752297 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:39.752310 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:39.752384 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:39.752737 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.753309 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.754069 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.754275 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.754335 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:39.757272 (+ 2937us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:52183: BlockingRecv error: recv got EOF from 127.1.116.129:52183 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":135,"threads_started":1}
Aug 07 00:40:39 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:39 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:39.898672 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:39.893325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:53929)
0807 00:40:39.893590 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:39.893605 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:39.893676 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:39.893839 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:39.893840 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:39.894033 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:39.894081 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:39.894083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.894842 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.894843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:39.895375 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:39.895376 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.895765 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.895766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:39.895821 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:39.896350 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:39.896354 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:39.898580 (+ 2226us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":154,"threads_started":1}
W20260807 00:40:39.898727 14888 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:39.893435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:53929 (local address 127.1.116.190:38399)
0807 00:40:39.893571 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:39.893574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:39.893649 (+ 75us) server_negotiation.cc:407] Connection header received
0807 00:40:39.893744 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:39.893744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:39.893758 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:39.893806 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:39.894176 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.894759 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.895472 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:39.895682 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:39.895767 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:39.898678 (+ 2911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:53929: BlockingRecv error: recv got EOF from 127.1.116.130:53929 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 07 00:40:40 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:40 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:40.037989 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:40.031986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:41991)
0807 00:40:40.032297 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:40.032311 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:40.032406 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:40.032559 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:40.032561 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:40.032804 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:40.032868 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:40.032870 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.033832 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.033835 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:40.034412 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:40.034415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.034887 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.034888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:40.034946 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:40.035544 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:40.035549 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:40.037914 (+ 2365us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":163,"threads_started":1}
W20260807 00:40:40.038111 14888 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:40.032141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:41991 (local address 127.1.116.190:38399)
0807 00:40:40.032284 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:40.032288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:40.032408 (+ 120us) server_negotiation.cc:407] Connection header received
0807 00:40:40.032476 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:40.032477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:40.032495 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:40.032553 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:40.032962 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.033660 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.034557 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.034777 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.034884 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:40.038031 (+ 3147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:41991: BlockingRecv error: recv got EOF from 127.1.116.131:41991 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 07 00:40:40 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:40 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:40.763903 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:40.757925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:35133)
0807 00:40:40.758094 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:40.758110 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:40.758215 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:40.758477 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:40.758478 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:40.758720 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:40.758793 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:40.758795 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.759536 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.759538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:40.760590 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:40.760593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.760992 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.760993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:40.761044 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:40.761678 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:40.761684 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:40.763825 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":92,"threads_started":1}
W20260807 00:40:40.764083 14892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:40.758107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:35133 (local address 127.1.116.190:38399)
0807 00:40:40.758297 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:40.758300 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:40.758314 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:40:40.758343 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:40.758344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:40.758358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:40.758445 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:40.758881 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.759407 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.760677 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.760880 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.760951 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:40.763955 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:35133: BlockingRecv error: recv got EOF from 127.1.116.129:35133 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":128,"threads_started":1}
Aug 07 00:40:40 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:40 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:40.905004 14896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:40.899301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:34133)
0807 00:40:40.899510 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:40.899527 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:40.899594 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:40.899789 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:40.899790 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:40.899975 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:40.900026 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:40.900027 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.900857 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.900858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:40.901379 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:40.901380 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.901767 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.901768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:40.901824 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:40.902391 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:40.902396 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:40.904909 (+ 2513us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":111,"threads_started":1}
W20260807 00:40:40.905071 14892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:40.899454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:34133 (local address 127.1.116.190:38399)
0807 00:40:40.899558 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:40.899560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:40.899573 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:40.899684 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:40.899684 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:40.899699 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:40.899784 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:40.900112 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.900766 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.901474 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:40.901672 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:40.901767 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:40.905019 (+ 3252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:34133: BlockingRecv error: recv got EOF from 127.1.116.130:34133 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 00:40:41 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:41 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:41.044075 14897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:41.038776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:38721)
0807 00:40:41.038971 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:41.038982 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:41.039052 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:41.039231 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:41.039232 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:41.039439 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:41.039508 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:41.039509 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.040224 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.040225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:41.040861 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:41.040863 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.041311 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.041312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:41.041360 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:41.041952 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:41.041958 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:41.043991 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
W20260807 00:40:41.044359 14892 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:41.038861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:38721 (local address 127.1.116.190:38399)
0807 00:40:41.038956 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:41.038958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:41.039057 (+ 99us) server_negotiation.cc:407] Connection header received
0807 00:40:41.039128 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:41.039129 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:41.039142 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:41.039193 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:41.039615 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.040119 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.041014 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.041221 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.041288 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:41.044279 (+ 2991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:38721: BlockingRecv error: recv got EOF from 127.1.116.131:38721 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 07 00:40:41 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:41 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:41.770437 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:41.764562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:35021)
0807 00:40:41.764734 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:41.764744 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:41.764807 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:41.765122 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:41.765123 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:41.765296 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:41.765355 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:41.765356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.766254 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.766255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:41.766992 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:41.766994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.767545 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.767546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:41.767610 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:41.768150 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:41.768155 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:41.770300 (+ 2145us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":101,"threads_started":1}
W20260807 00:40:41.770478 14899 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:41.764772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:35021 (local address 127.1.116.190:38399)
0807 00:40:41.764946 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:41.764948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:41.764958 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:41.764987 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:41.764987 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:41.765002 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:41.765092 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:41.765438 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.766114 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.767081 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.767444 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.767525 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:41.770391 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:35021: BlockingRecv error: recv got EOF from 127.1.116.129:35021 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
Aug 07 00:40:41 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:41 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:41.911338 14900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:41.905791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:49919)
0807 00:40:41.906017 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:41.906035 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:41.906146 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:41.906325 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:41.906326 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:41.906551 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:41.906602 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:41.906603 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.907383 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.907384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:41.907905 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:41.907907 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.908301 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.908302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:41.908355 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:41.908899 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:41.908903 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:41.911237 (+ 2334us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":134,"threads_started":1}
W20260807 00:40:41.911383 14899 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:41.905925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:49919 (local address 127.1.116.190:38399)
0807 00:40:41.906023 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:41.906026 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:41.906124 (+ 98us) server_negotiation.cc:407] Connection header received
0807 00:40:41.906223 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:41.906223 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:41.906237 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:41.906315 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:41.906698 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.907302 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.908001 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:41.908221 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:41.908303 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:41.911320 (+ 3017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:49919: BlockingRecv error: recv got EOF from 127.1.116.130:49919 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 00:40:42 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:42 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:42.050817 14901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:42.044737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:51809)
0807 00:40:42.044942 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:42.044955 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:42.045077 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:42.045273 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:42.045274 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:42.045475 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:42.045526 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:42.045527 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.046210 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.046210 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:42.046917 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:42.046920 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.047559 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.047561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:42.047640 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:42.048295 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:42.048301 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:42.050743 (+ 2442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":112,"threads_started":1}
W20260807 00:40:42.050999 14899 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:42.044786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51809 (local address 127.1.116.190:38399)
0807 00:40:42.044868 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:42.044871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:42.045046 (+ 175us) server_negotiation.cc:407] Connection header received
0807 00:40:42.045176 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:42.045177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:42.045197 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:42.045263 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:42.045617 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.046119 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.047062 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.047393 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.047556 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:42.050884 (+ 3328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:51809: BlockingRecv error: recv got EOF from 127.1.116.131:51809 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 07 00:40:42 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:42 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:42.780748 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:42.771015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:41049)
0807 00:40:42.771298 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:42.771314 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:42.771409 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:42.771570 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:42.771571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:42.771792 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:42.771865 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:42.771867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.772557 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.772558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:42.773162 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:42.773164 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.773518 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.773518 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:42.773561 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:42.778637 (+ 5076us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:42.778642 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:42.780676 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":146,"threads_started":1}
W20260807 00:40:42.780942 14903 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:42.771217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:41049 (local address 127.1.116.190:38399)
0807 00:40:42.771412 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:42.771414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:42.771424 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:40:42.771456 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:42.771457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:42.771469 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:42.771539 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:42.771955 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.772467 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.773252 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.773441 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.773499 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:42.780834 (+ 7335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:41049: BlockingRecv error: recv got EOF from 127.1.116.129:41049 (error 108)
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":101,"threads_started":1}
Aug 07 00:40:42 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:42 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:42.917187 14904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:42.912083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:48897)
0807 00:40:42.912328 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:42.912339 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:42.912414 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:42.912609 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:42.912610 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:42.912826 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:42.912910 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:42.912912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.913574 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.913575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:42.914147 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:42.914148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.914492 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.914493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:42.914612 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:42.915197 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:42.915202 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:42.917116 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":172,"threads_started":1}
W20260807 00:40:42.917364 14903 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:42.912171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:48897 (local address 127.1.116.190:38399)
0807 00:40:42.912256 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:42.912258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:42.912419 (+ 161us) server_negotiation.cc:407] Connection header received
0807 00:40:42.912475 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:42.912476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:42.912487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:42.912538 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:42.912998 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.913494 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.914235 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:42.914414 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:42.914474 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:42.917268 (+ 2794us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:48897: BlockingRecv error: recv got EOF from 127.1.116.130:48897 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 00:40:43 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:43 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:43.056938 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:43.051457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:37315)
0807 00:40:43.051744 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:43.051757 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:43.051834 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:43.052073 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:43.052075 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:43.052378 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:43.052449 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:43.052451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.053245 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.053246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:43.053827 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:43.053828 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.054235 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.054236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:43.054281 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:43.054926 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:43.054932 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:43.056856 (+ 1924us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":117,"threads_started":1}
W20260807 00:40:43.057192 14903 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:43.051618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:37315 (local address 127.1.116.190:38399)
0807 00:40:43.051726 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:43.051728 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:43.051867 (+ 139us) server_negotiation.cc:407] Connection header received
0807 00:40:43.051905 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:43.051906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:43.051927 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:43.052012 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:43.052599 (+ 587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.053126 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.053954 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.054149 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.054217 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:43.057101 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:37315: BlockingRecv error: recv got EOF from 127.1.116.131:37315 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 07 00:40:43 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:43 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:43.787330 14906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:43.781325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:40517)
0807 00:40:43.781513 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:43.781526 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:43.781589 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:43.781859 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:43.781860 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:43.782126 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:43.782208 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:43.782210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.783474 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.783475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:43.784311 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:43.784314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.784733 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.784734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:43.784791 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:43.785334 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:43.785340 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:43.787216 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":76,"threads_started":1}
W20260807 00:40:43.787362 14907 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:43.781419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:40517 (local address 127.1.116.190:38399)
0807 00:40:43.781637 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:43.781639 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:43.781646 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:40:43.781695 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:43.781696 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:43.781711 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:43.781833 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:43.782305 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.783268 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.784411 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.784646 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.784734 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:43.787289 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:40517: BlockingRecv error: recv got EOF from 127.1.116.129:40517 (error 108)
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":97,"threads_started":1}
Aug 07 00:40:43 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:43 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:43.922736 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:43.917715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:42347)
0807 00:40:43.917863 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:43.917875 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:43.917935 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:43.918102 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:43.918103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:43.918290 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:43.918343 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:43.918345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.919116 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.919117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:43.919685 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:43.919686 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.920037 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.920038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:43.920080 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:43.920574 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:43.920578 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:43.922662 (+ 2084us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":77,"threads_started":1}
W20260807 00:40:43.922926 14907 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:43.917787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:42347 (local address 127.1.116.190:38399)
0807 00:40:43.917896 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:43.917899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:43.917915 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:40:43.918015 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:43.918016 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:43.918030 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:43.918088 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:43.918417 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.919037 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.919770 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:43.919960 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:43.920019 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:43.922804 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:42347: BlockingRecv error: recv got EOF from 127.1.116.130:42347 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 07 00:40:44 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:44 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:44.062917 14912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:44.057492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:57017)
0807 00:40:44.057792 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:44.057804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:44.057914 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:44.058120 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:44.058121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:44.058513 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:44.058636 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:44.058638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.059315 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.059316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:44.059908 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:44.059910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.060249 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.060249 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:44.060300 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:44.060897 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:44.060903 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:44.062835 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":130,"threads_started":1}
W20260807 00:40:44.063098 14907 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:44.057592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:57017 (local address 127.1.116.190:38399)
0807 00:40:44.057672 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:44.057675 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:44.057911 (+ 236us) server_negotiation.cc:407] Connection header received
0807 00:40:44.057983 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:44.057984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:44.058010 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:44.058087 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:44.058715 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.059235 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.059986 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.060172 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.060227 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:44.063002 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:57017: BlockingRecv error: recv got EOF from 127.1.116.131:57017 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 00:40:44 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:44 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:44.794436 14914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:44.788051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:46211)
0807 00:40:44.788251 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:44.788264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:44.788330 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:44.788548 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:44.788549 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:44.788766 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:44.788836 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:44.788839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.789976 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.789977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:44.790741 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:44.790744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.791208 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.791210 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:44.791273 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:44.791973 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:44.791979 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:44.794320 (+ 2341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":124,"threads_started":1}
W20260807 00:40:44.794499 14915 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:44.788149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:46211 (local address 127.1.116.190:38399)
0807 00:40:44.788311 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:44.788313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:44.788328 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:40:44.788401 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:44.788402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:44.788433 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:44.788536 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:44.788932 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.789859 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.790838 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.791074 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.791198 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:44.794421 (+ 3223us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:46211: BlockingRecv error: recv got EOF from 127.1.116.129:46211 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":82,"threads_started":1}
Aug 07 00:40:44 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:44 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:44.928432 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:44.923388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:56433)
0807 00:40:44.923560 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:44.923575 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:44.923660 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:44.923894 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:44.923896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:44.924103 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:44.924157 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:44.924159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.924950 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.924951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:44.925492 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:44.925493 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.925882 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.925883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:44.925924 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:44.926452 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:44.926456 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:44.928360 (+ 1904us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":77,"threads_started":1}
W20260807 00:40:44.928560 14915 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:44.923476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:56433 (local address 127.1.116.190:38399)
0807 00:40:44.923580 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:44.923583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:44.923628 (+ 45us) server_negotiation.cc:407] Connection header received
0807 00:40:44.923760 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:44.923762 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:44.923778 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:44.923862 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:44.924282 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.924865 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.925593 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:44.925796 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:44.925871 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:44.928471 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:56433: BlockingRecv error: recv got EOF from 127.1.116.130:56433 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 07 00:40:45 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:45 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:45.069314 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:45.063466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:42695)
0807 00:40:45.063747 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:45.063764 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:45.063841 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:45.064002 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:45.064003 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:45.064223 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:45.064289 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:45.064291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.065042 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.065043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:45.065642 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:45.065644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.066190 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.066191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:45.066275 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:45.066910 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:45.066917 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:45.069201 (+ 2284us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":206,"threads_started":1}
W20260807 00:40:45.069362 14915 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:45.063578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:42695 (local address 127.1.116.190:38399)
0807 00:40:45.063714 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:45.063716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:45.063843 (+ 127us) server_negotiation.cc:407] Connection header received
0807 00:40:45.063907 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:45.063908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:45.063920 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:45.063970 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:45.064395 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.064934 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.065849 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.066060 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.066144 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:45.069291 (+ 3147us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:42695: BlockingRecv error: recv got EOF from 127.1.116.131:42695 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 07 00:40:45 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:45 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:45.800213 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:45.795185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:38525)
0807 00:40:45.795437 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:45.795448 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:45.795503 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:45.795712 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:45.795713 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:45.795896 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:45.795964 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:45.795965 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.796722 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.796723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:45.797276 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:45.797277 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.797672 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.797673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:45.797714 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:45.798290 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:45.798295 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:45.800152 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":182,"threads_started":1}
W20260807 00:40:45.800400 14921 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:45.795288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:38525 (local address 127.1.116.190:38399)
0807 00:40:45.795491 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:45.795493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:45.795508 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:40:45.795567 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:45.795568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:45.795582 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:45.795683 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:45.796058 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.796638 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.797373 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.797573 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.797657 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:45.800280 (+ 2623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:38525: BlockingRecv error: recv got EOF from 127.1.116.129:38525 (error 108)
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":129,"threads_started":1}
Aug 07 00:40:45 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:45 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:45.934167 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:45.929057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:43861)
0807 00:40:45.929256 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:45.929268 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:45.929323 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:45.929479 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:45.929479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:45.929651 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:45.929708 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:45.929710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.930447 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.930447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:45.931029 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:45.931031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.931406 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.931406 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:45.931447 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:45.931966 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:45.931970 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:45.934096 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":115,"threads_started":1}
W20260807 00:40:45.934319 14921 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:45.929214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:43861 (local address 127.1.116.190:38399)
0807 00:40:45.929306 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:45.929308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:45.929321 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:45.929388 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:45.929389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:45.929401 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:45.929460 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:45.929787 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.930354 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.931117 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:45.931329 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:45.931388 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:45.934229 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:43861: BlockingRecv error: recv got EOF from 127.1.116.130:43861 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 00:40:46 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:46 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:46.075583 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:46.069961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:41447)
0807 00:40:46.070235 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:46.070251 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:46.070359 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:46.070581 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:46.070583 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:46.070789 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:46.070838 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:46.070840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.071553 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.071554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:46.072097 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:46.072098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.072465 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.072466 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:46.072521 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:46.073089 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:46.073094 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:46.075495 (+ 2401us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":151,"threads_started":1}
W20260807 00:40:46.075793 14921 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:46.070127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:41447 (local address 127.1.116.190:38399)
0807 00:40:46.070236 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:46.070239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:46.070336 (+ 97us) server_negotiation.cc:407] Connection header received
0807 00:40:46.070430 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:46.070432 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:46.070447 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:46.070535 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:46.070948 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.071452 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.072190 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.072381 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.072463 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:46.075623 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:41447: BlockingRecv error: recv got EOF from 127.1.116.131:41447 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 00:40:46 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:46 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:46.806123 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:46.800769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:41719)
0807 00:40:46.800960 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:46.800973 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:46.801056 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:46.801398 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:46.801399 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:46.801618 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:46.801692 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:46.801694 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.802436 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.802437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:46.803130 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:46.803132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.803476 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.803477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:46.803521 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:46.804116 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:46.804122 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:46.806038 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":113,"threads_started":1}
W20260807 00:40:46.806324 14929 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:46.800846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:41719 (local address 127.1.116.190:38399)
0807 00:40:46.800993 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:46.800995 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:46.801042 (+ 47us) server_negotiation.cc:407] Connection header received
0807 00:40:46.801123 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:46.801124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:46.801135 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:46.801211 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:46.801778 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.802314 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.803211 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.803399 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.803461 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:46.806146 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:41719: BlockingRecv error: recv got EOF from 127.1.116.129:41719 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":86,"threads_started":1}
Aug 07 00:40:46 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:46 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:46.939790 14930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:46.934731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:33721)
0807 00:40:46.934970 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:46.934986 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:46.935103 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:46.935262 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:46.935263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:46.935481 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:46.935529 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:46.935530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.936216 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.936217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:46.936751 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:46.936753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.937118 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.937119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:46.937176 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:46.937714 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:46.937719 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:46.939678 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":115,"threads_started":1}
W20260807 00:40:46.939818 14929 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:46.934847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:33721 (local address 127.1.116.190:38399)
0807 00:40:46.934964 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:46.934967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:46.935073 (+ 106us) server_negotiation.cc:407] Connection header received
0807 00:40:46.935165 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:46.935166 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:46.935180 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:46.935259 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:46.935617 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.936122 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.936844 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:46.937040 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:46.937119 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:46.939755 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:33721: BlockingRecv error: recv got EOF from 127.1.116.130:33721 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 07 00:40:47 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:47 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.131@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:47.081354 14931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:47.076112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.131:52743)
0807 00:40:47.076378 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:47.076398 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:47.076498 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:47.076675 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:47.076676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:47.076869 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:47.076916 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:47.076917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.077662 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.077663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:47.078202 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:47.078204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.078762 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.078763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:47.078807 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:47.079341 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:47.079346 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:47.081291 (+ 1945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":156,"threads_started":1}
W20260807 00:40:47.081506 14929 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:47.076238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:52743 (local address 127.1.116.190:38399)
0807 00:40:47.076356 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:47.076358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:47.076465 (+ 107us) server_negotiation.cc:407] Connection header received
0807 00:40:47.076572 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:47.076573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:47.076587 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:47.076655 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:47.077002 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.077580 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.078388 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.078673 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.078756 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:47.081407 (+ 2651us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.131:52743: BlockingRecv error: recv got EOF from 127.1.116.131:52743 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 00:40:47 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:47 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.129@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:47.816900 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:47.806738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.129:38963)
0807 00:40:47.806967 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:47.806984 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:47.807035 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:47.807387 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:47.807388 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:47.807569 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:47.807628 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:47.807630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.808743 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.808744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:47.809292 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:47.809293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.809721 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.809722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:47.809763 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:47.815088 (+ 5325us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:47.815093 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:47.816840 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":172,"threads_started":1}
W20260807 00:40:47.817338 14934 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:47.806976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:38963 (local address 127.1.116.190:38399)
0807 00:40:47.807162 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:47.807165 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:47.807177 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:40:47.807219 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:47.807220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:47.807235 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:47.807382 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:47.807801 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.808659 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.809399 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.809630 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.809709 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:47.817253 (+ 7544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.129:38963: BlockingRecv error: recv got EOF from 127.1.116.129:38963 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":109,"threads_started":1}
Aug 07 00:40:47 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:40:47 dist-test-slave-h6n0 krb5kdc[13425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.116.130@KRBTEST.COM for oryx/127.1.116.190@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:40:47.945425 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:47.940493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38399 (local address 127.1.116.130:34571)
0807 00:40:47.940666 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:47.940680 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:47.940746 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:47.940953 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:47.940954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:47.941173 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:47.941239 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:47.941241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.942036 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.942037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:47.942654 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:47.942655 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.943036 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.943037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:47.943094 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:47.943616 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:47.943620 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:47.945366 (+ 1746us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.116.190:38399: Server oryx/127.1.116.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
W20260807 00:40:47.945621 14934 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:47.940632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:34571 (local address 127.1.116.190:38399)
0807 00:40:47.940710 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:47.940713 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:47.940728 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:40:47.940841 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:47.940842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:47.940858 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:47.940917 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:47.941324 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.941958 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.942742 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:47.942932 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:47.942992 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:47.945517 (+ 2525us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.116.130:34571: BlockingRecv error: recv got EOF from 127.1.116.130:34571 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:40:47.984748 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 14261
I20260807 00:40:47.997277 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 14480
I20260807 00:40:48.012903 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 14670
I20260807 00:40:48.026453 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 14161
2026-08-07T00:40:48Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17792 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T00:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:48Z Disabled control of system clock
I20260807 00:40:48.064958 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39167
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 00:40:48.152766 14941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:48.152971 14941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:48.152993 14941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:48.154220 14941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:48.154268 14941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:48.154280 14941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:48.154294 14941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:48.154306 14941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:48.155716 14941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:48.155988 14941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:48.156327 14941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:48.161793 14950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:48.161796 14949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.162238 14941 server_base.cc:1034] running on GCE node
W20260807 00:40:48.161793 14953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.162711 14941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:48.163009 14941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:48.164189 14941 hybrid_clock.cc:648] HybridClock initialized: now 1786063248164160 us; error 51 us; skew 500 ppm
I20260807 00:40:48.165676 14941 webserver.cc:460] Webserver started at http://127.1.116.190:35221/ using document root <none> and password file <none>
I20260807 00:40:48.166049 14941 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:48.166261 14941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:48.166471 14941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:48.167870 14941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "ccf98c6414fa4492accdc4ed3e33e64f"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "2f421e934ea6d5a3ea890122fd99dd78"
server_key_iv: "6a89ab957711e40c11abda94f3cda05a"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.168541 14941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "ccf98c6414fa4492accdc4ed3e33e64f"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "2f421e934ea6d5a3ea890122fd99dd78"
server_key_iv: "6a89ab957711e40c11abda94f3cda05a"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.170050 14941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260807 00:40:48.171118 14960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.171411 14941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:48.171530 14941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "ccf98c6414fa4492accdc4ed3e33e64f"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "2f421e934ea6d5a3ea890122fd99dd78"
server_key_iv: "6a89ab957711e40c11abda94f3cda05a"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.171622 14941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:48.186689 14941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:48.186954 14941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:48.187116 14941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:48.191880 14941 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:39167
I20260807 00:40:48.191973 15031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:39167 every 8 connection(s)
I20260807 00:40:48.192333 14941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:48.193219 15032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:48.196707 15032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f: Bootstrap starting.
I20260807 00:40:48.197445 15032 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:48.197816 15032 log.cc:824] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:48.198736 15032 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f: No bootstrap required, opened a new log
I20260807 00:40:48.200485 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 14941
I20260807 00:40:48.200580 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:48.200657 15032 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39167 } }
I20260807 00:40:48.200807 1490 external_mini_cluster.cc:1428] Setting key 056834b9648cff89c0a32b08d7b3f752
I20260807 00:40:48.200862 15032 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:48.200915 15032 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccf98c6414fa4492accdc4ed3e33e64f, State: Initialized, Role: FOLLOWER
I20260807 00:40:48.201118 15032 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39167 } }
I20260807 00:40:48.201238 15032 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:48.201278 15032 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:48.201341 15032 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:48.202247 15032 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39167 } }
I20260807 00:40:48.202370 15032 leader_election.cc:302] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccf98c6414fa4492accdc4ed3e33e64f; no voters:
I20260807 00:40:48.202574 15032 leader_election.cc:288] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:48.202878 15040 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:48.202901 15032 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:48.203083 15040 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [term 1 LEADER]: Becoming Leader. State: Replica: ccf98c6414fa4492accdc4ed3e33e64f, State: Running, Role: LEADER
I20260807 00:40:48.203197 15040 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39167 } }
I20260807 00:40:48.203629 15040 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccf98c6414fa4492accdc4ed3e33e64f. Latest consensus state: current_term: 1 leader_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39167 } } }
I20260807 00:40:48.203725 15040 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:48.204077 15043 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccf98c6414fa4492accdc4ed3e33e64f" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39167 } } }
I20260807 00:40:48.204156 15043 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:48.204538 15057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:48.205185 15057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:48.207270 15057 catalog_manager.cc:1347] Generated new cluster ID: f4177473d3fd41cabd882a7c3ab0982b
I20260807 00:40:48.207337 15057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:48.207894 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.201436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42098 (local address 127.1.116.190:39167)
0807 00:40:48.201717 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:48.201729 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:48.201802 (+ 73us) server_negotiation.cc:407] Connection header received
0807 00:40:48.201947 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:48.201956 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:48.202050 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:48.202219 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:48.202512 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.204174 (+ 1662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.205437 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.206112 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.206263 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:48.207136 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:48.207140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:48.207141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:48.207385 (+ 244us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:48.207427 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:48.207443 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:48.207521 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:48.207615 (+ 94us) server_negotiation.cc:299] Negotiation successful
0807 00:40:48.207694 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":94,"threads_started":1}
I20260807 00:40:48.212927 15057 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:48.213604 15057 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:48.220065 15057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccf98c6414fa4492accdc4ed3e33e64f: Generated new TSK 0
I20260807 00:40:48.220475 15057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:40:48.223945 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39167
--builtin_ntp_servers=127.1.116.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 00:40:48.327191 15070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:48.327543 15070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:48.327584 15070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:48.329519 15070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:48.329578 15070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:48.329612 15070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:48.331202 15070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:48.331542 15070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:48.331925 15070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:48.340175 15079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.340721 15070 server_base.cc:1034] running on GCE node
W20260807 00:40:48.340310 15082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:48.340510 15080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.341113 15070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:48.341435 15070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:48.342592 15070 hybrid_clock.cc:648] HybridClock initialized: now 1786063248342568 us; error 37 us; skew 500 ppm
I20260807 00:40:48.344486 15070 webserver.cc:460] Webserver started at http://127.1.116.129:34181/ using document root <none> and password file <none>
I20260807 00:40:48.344851 15070 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:48.344934 15070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:48.345093 15070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:48.346812 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "f57d7f257b5a49428e98ea1f2fcb04b5"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "97b8410b2a4649fb0535ed684f31cc9e"
server_key_iv: "8b42e69c267bbc97edd79cce5c3a3a84"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.347334 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "f57d7f257b5a49428e98ea1f2fcb04b5"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "97b8410b2a4649fb0535ed684f31cc9e"
server_key_iv: "8b42e69c267bbc97edd79cce5c3a3a84"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.349649 15070 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 00:40:48.350926 15091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.351198 15070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:48.351311 15070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "f57d7f257b5a49428e98ea1f2fcb04b5"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "97b8410b2a4649fb0535ed684f31cc9e"
server_key_iv: "8b42e69c267bbc97edd79cce5c3a3a84"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.351402 15070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:48.384904 15070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:48.385392 15070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:48.385581 15070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:48.385905 15070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:48.386339 15070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:48.386377 15070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.386404 15070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:48.386416 15070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.395874 15070 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:41029
I20260807 00:40:48.395925 15248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:41029 every 8 connection(s)
I20260807 00:40:48.396353 15070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:48.402711 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 15070
I20260807 00:40:48.402828 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:48.403026 1490 external_mini_cluster.cc:1428] Setting key bd926b21006c63d12f1fc742651be6b4
I20260807 00:40:48.404650 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.397390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:46459 (local address 127.1.116.190:39167)
0807 00:40:48.397491 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:48.397494 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:48.397750 (+ 256us) server_negotiation.cc:407] Connection header received
0807 00:40:48.398406 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:48.398408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:48.398427 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:48.398497 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:48.399755 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.400568 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.401991 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.402516 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.402633 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:48.404090 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:48.404092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:48.404092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:48.404098 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:48.404210 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:48.404224 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:48.404312 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:48.404517 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 00:40:48.404557 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:40:48.404752 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39167
--builtin_ntp_servers=127.1.116.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 00:40:48.404953 15254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.397264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39167 (local address 127.1.116.129:46459)
0807 00:40:48.397589 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:48.397631 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:48.398265 (+ 634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:48.398692 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:48.398698 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:48.398713 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:48.399579 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:48.399592 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.400745 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.400746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:48.401853 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:48.401856 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.402753 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.402754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:48.402983 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:48.403940 (+ 957us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:48.403941 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:48.403979 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:48.404008 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:48.404357 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:48.404366 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:48.404371 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:40:48.404494 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 00:40:48.404643 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":112,"threads_started":1}
I20260807 00:40:48.405557 15250 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39167
I20260807 00:40:48.405692 15250 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:48.405925 15250 heartbeater.cc:511] Master 127.1.116.190:39167 requested a full tablet report, sending...
I20260807 00:40:48.406639 14986 ts_manager.cc:194] Registered new tserver with Master: f57d7f257b5a49428e98ea1f2fcb04b5 (127.1.116.129:41029)
I20260807 00:40:48.407454 14986 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.129:46459
W20260807 00:40:48.498769 15255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:48.499004 15255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:48.499027 15255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:48.500341 15255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:48.500396 15255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:48.500428 15255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:48.501965 15255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:48.502288 15255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:48.502725 15255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:48.508670 15262 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:48.508781 15267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:48.508935 15264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.508852 15255 server_base.cc:1034] running on GCE node
I20260807 00:40:48.509266 15255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:48.509574 15255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:48.510756 15255 hybrid_clock.cc:648] HybridClock initialized: now 1786063248510756 us; error 96 us; skew 500 ppm
I20260807 00:40:48.512319 15255 webserver.cc:460] Webserver started at http://127.1.116.130:44625/ using document root <none> and password file <none>
I20260807 00:40:48.512645 15255 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:48.512740 15255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:48.512895 15255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:48.514124 15255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "15ce522b01b449939fbd560aac6c3c9f"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "4fe4bdc6de61693231187acf94ddd127"
server_key_iv: "f51ddfb71d4b5602a6275e6b076981f8"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.514595 15255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "15ce522b01b449939fbd560aac6c3c9f"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "4fe4bdc6de61693231187acf94ddd127"
server_key_iv: "f51ddfb71d4b5602a6275e6b076981f8"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.516126 15255 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:48.517107 15277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.517369 15255 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:48.517483 15255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "15ce522b01b449939fbd560aac6c3c9f"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "4fe4bdc6de61693231187acf94ddd127"
server_key_iv: "f51ddfb71d4b5602a6275e6b076981f8"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.517576 15255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:48.530037 15255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:48.530370 15255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:48.530580 15255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:48.530864 15255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:48.531250 15255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:48.531309 15255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.531347 15255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:48.531375 15255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.539021 15255 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:33755
I20260807 00:40:48.539069 15431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:33755 every 8 connection(s)
I20260807 00:40:48.539446 15255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:48.542428 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 15255
I20260807 00:40:48.542573 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:48.542814 1490 external_mini_cluster.cc:1428] Setting key 65ce97ecf44b43181b3250e5bef7fb0d
I20260807 00:40:48.543977 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39167
--builtin_ntp_servers=127.1.116.148:36551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 00:40:48.545823 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.540334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:42953 (local address 127.1.116.190:39167)
0807 00:40:48.540450 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:48.540453 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:48.540755 (+ 302us) server_negotiation.cc:407] Connection header received
0807 00:40:48.541319 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:48.541320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:48.541332 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:48.541396 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:48.542074 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.542716 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.544239 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.544429 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.544491 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:48.545487 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:48.545489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:48.545489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:48.545493 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:48.545509 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:48.545513 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:48.545561 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:48.545682 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 00:40:48.545724 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 00:40:48.546084 15434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.540289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39167 (local address 127.1.116.130:42953)
0807 00:40:48.540589 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:48.540647 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:48.541215 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:48.541455 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:48.541459 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:48.541468 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:48.541996 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:48.542000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.542822 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.542823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:48.544141 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:48.544144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.544530 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.544532 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:48.544699 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:48.545321 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:48.545322 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:48.545361 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:48.545391 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:48.545590 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:48.545595 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:48.545598 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:48.545673 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:40:48.545789 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":126,"threads_started":1}
I20260807 00:40:48.546748 15432 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39167
I20260807 00:40:48.546839 15432 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:48.547006 15432 heartbeater.cc:511] Master 127.1.116.190:39167 requested a full tablet report, sending...
I20260807 00:40:48.547420 14986 ts_manager.cc:194] Registered new tserver with Master: 15ce522b01b449939fbd560aac6c3c9f (127.1.116.130:33755)
I20260807 00:40:48.547843 14986 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.130:42953
W20260807 00:40:48.636650 15436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:48.636926 15436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:48.636948 15436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:48.638260 15436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:48.638317 15436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:48.638345 15436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:48.639839 15436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:36551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:48.640151 15436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:48.640544 15436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:48.646420 15441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:48.646435 15443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.646667 15436 server_base.cc:1034] running on GCE node
W20260807 00:40:48.646731 15446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.647066 15436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:48.647369 15436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:48.648550 15436 hybrid_clock.cc:648] HybridClock initialized: now 1786063248648528 us; error 49 us; skew 500 ppm
I20260807 00:40:48.650125 15436 webserver.cc:460] Webserver started at http://127.1.116.131:39915/ using document root <none> and password file <none>
I20260807 00:40:48.650414 15436 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:48.650491 15436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:48.650678 15436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:48.651942 15436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "838edb245513407fa7225414a3e4000e"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "dbf082fd36f216508c01f1e9e4df66f0"
server_key_iv: "1895fbc3aa70fb1aecb2a753b16cbc7a"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.652369 15436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "838edb245513407fa7225414a3e4000e"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "dbf082fd36f216508c01f1e9e4df66f0"
server_key_iv: "1895fbc3aa70fb1aecb2a753b16cbc7a"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.653847 15436 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:40:48.654791 15455 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.655050 15436 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:48.655148 15436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "838edb245513407fa7225414a3e4000e"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "dbf082fd36f216508c01f1e9e4df66f0"
server_key_iv: "1895fbc3aa70fb1aecb2a753b16cbc7a"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.655241 15436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:48.672823 15436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:48.673085 15436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:48.673189 15436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:48.673370 15436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:48.673818 15436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:48.673863 15436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.673887 15436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:48.673899 15436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.680886 15436 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:37107
I20260807 00:40:48.681025 15609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:37107 every 8 connection(s)
I20260807 00:40:48.681365 15436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:48.687739 15039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.682257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:59315 (local address 127.1.116.190:39167)
0807 00:40:48.682371 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:48.682375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:48.682746 (+ 371us) server_negotiation.cc:407] Connection header received
0807 00:40:48.683337 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:48.683339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:48.683353 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:48.683419 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:48.684123 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.684710 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.686065 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.686272 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.686338 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:48.687447 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:48.687449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:48.687449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:48.687453 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:48.687470 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:48.687474 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:48.687528 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:48.687630 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 00:40:48.687663 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:40:48.687930 15615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.682227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39167 (local address 127.1.116.131:59315)
0807 00:40:48.682593 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:48.682642 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:48.683182 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:48.683484 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:48.683488 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:48.683496 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:48.684001 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:48.684006 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.684865 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.684867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:48.685951 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:48.685956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:48.686409 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:48.686411 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:48.686649 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:48.687285 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:48.687286 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:48.687315 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:48.687339 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:48.687549 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:48.687553 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:48.687555 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:40:48.687613 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 00:40:48.687713 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":180,"threads_started":1}
I20260807 00:40:48.688512 15612 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39167
I20260807 00:40:48.688660 15612 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:48.688915 15612 heartbeater.cc:511] Master 127.1.116.190:39167 requested a full tablet report, sending...
I20260807 00:40:48.689440 14986 ts_manager.cc:194] Registered new tserver with Master: 838edb245513407fa7225414a3e4000e (127.1.116.131:37107)
I20260807 00:40:48.689869 14986 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.131:59315
I20260807 00:40:48.691673 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 15436
I20260807 00:40:48.691771 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:48.691993 1490 external_mini_cluster.cc:1428] Setting key f1daa8d71cd83c7aa62bdbc3cef54cda
I20260807 00:40:48.693154 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 00:40:48.695664 15039 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:48.694864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42106 (local address 127.1.116.190:39167)
0807 00:40:48.694938 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:48.694940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:48.695073 (+ 133us) server_negotiation.cc:407] Connection header received
0807 00:40:48.695193 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:48.695194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:48.695207 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:48.695272 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:48.695582 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42106: BlockingRecv error: recv got EOF from 127.0.0.1:42106 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 00:40:48.695726 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 15070
I20260807 00:40:48.712373 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 15255
I20260807 00:40:48.725072 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 15436
I20260807 00:40:48.732959 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 14941
2026-08-07T00:40:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (704 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T00:40:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:48Z Disabled control of system clock
I20260807 00:40:48.770990 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38967
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:38967
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 00:40:48.866923 15627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:48.867156 15627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:48.867185 15627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:48.868403 15627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:48.868449 15627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:48.868462 15627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:48.868475 15627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:48.868536 15627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:48.869956 15627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:38967
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:38967
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:48.870252 15627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:48.870656 15627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:48.876243 15637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:48.876245 15633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:48.876243 15634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:48.876770 15627 server_base.cc:1034] running on GCE node
I20260807 00:40:48.877024 15627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:48.877493 15627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:48.878686 15627 hybrid_clock.cc:648] HybridClock initialized: now 1786063248878632 us; error 69 us; skew 500 ppm
I20260807 00:40:48.880286 15627 webserver.cc:460] Webserver started at http://127.1.116.190:40843/ using document root <none> and password file <none>
I20260807 00:40:48.880578 15627 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:48.880635 15627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:48.880792 15627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:48.881953 15627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "7f54eb0acccb40489590445ac701dde4"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "f4f50c4750ed99a6cd0fc0d7fb262854"
server_key_iv: "059aebd2c1d648618121c987b82aa374"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.882354 15627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "7f54eb0acccb40489590445ac701dde4"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "f4f50c4750ed99a6cd0fc0d7fb262854"
server_key_iv: "059aebd2c1d648618121c987b82aa374"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.883848 15627 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:48.885028 15645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:48.885279 15627 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:48.885387 15627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "7f54eb0acccb40489590445ac701dde4"
format_stamp: "Formatted at 2026-08-07 00:40:48 on dist-test-slave-h6n0"
server_key: "f4f50c4750ed99a6cd0fc0d7fb262854"
server_key_iv: "059aebd2c1d648618121c987b82aa374"
server_key_version: "encryptionkey@0"
I20260807 00:40:48.885474 15627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:48.889228 15627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:48.889497 15627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:48.889605 15627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:48.894758 15627 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:38967
I20260807 00:40:48.894842 15721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:38967 every 8 connection(s)
I20260807 00:40:48.895236 15627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:48.896148 15722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:48.896919 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 15627
I20260807 00:40:48.897006 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:48.897257 1490 external_mini_cluster.cc:1428] Setting key dedf266d7ac7b38ce725eafdd10c027e
I20260807 00:40:48.899957 15722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4: Bootstrap starting.
I20260807 00:40:48.900506 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:48.897987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.1.116.190:38967)
0807 00:40:48.898217 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:48.898226 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:48.898258 (+ 32us) server_negotiation.cc:407] Connection header received
0807 00:40:48.898938 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:48.898955 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:48.899005 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:48.899150 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:48.899946 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:48.899950 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:48.899951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:48.899955 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:48.900003 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:48.900014 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:48.900115 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:48.900266 (+ 151us) server_negotiation.cc:299] Negotiation successful
0807 00:40:48.900336 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260807 00:40:48.900774 15722 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:48.901105 15722 log.cc:824] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:48.901943 15722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4: No bootstrap required, opened a new log
I20260807 00:40:48.903432 15722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f54eb0acccb40489590445ac701dde4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38967 } }
I20260807 00:40:48.903596 15722 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:48.903647 15722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f54eb0acccb40489590445ac701dde4, State: Initialized, Role: FOLLOWER
I20260807 00:40:48.903797 15722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f54eb0acccb40489590445ac701dde4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38967 } }
I20260807 00:40:48.903908 15722 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:48.903954 15722 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:48.904009 15722 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:48.904533 15722 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f54eb0acccb40489590445ac701dde4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38967 } }
I20260807 00:40:48.904659 15722 leader_election.cc:302] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f54eb0acccb40489590445ac701dde4; no voters:
I20260807 00:40:48.904844 15722 leader_election.cc:288] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:48.905186 15722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:48.905232 15729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:48.905429 15729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [term 1 LEADER]: Becoming Leader. State: Replica: 7f54eb0acccb40489590445ac701dde4, State: Running, Role: LEADER
I20260807 00:40:48.905558 15729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f54eb0acccb40489590445ac701dde4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38967 } }
I20260807 00:40:48.906040 15729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f54eb0acccb40489590445ac701dde4. Latest consensus state: current_term: 1 leader_uuid: "7f54eb0acccb40489590445ac701dde4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f54eb0acccb40489590445ac701dde4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38967 } } }
I20260807 00:40:48.906141 15729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:48.906383 15730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f54eb0acccb40489590445ac701dde4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f54eb0acccb40489590445ac701dde4" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 38967 } } }
I20260807 00:40:48.906476 15730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4 [sys.catalog]: This master's current role is: LEADER
W20260807 00:40:48.907387 15748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:40:48.910408 15748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:40:48.910490 15741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:48.910774 15741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:48.913065 15741 catalog_manager.cc:1347] Generated new cluster ID: d6c13862936d4473b4813c967adc74af
I20260807 00:40:48.913130 15741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:40:48.925446 15741 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:48.925606 15741 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:48.931834 15741 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f54eb0acccb40489590445ac701dde4: Generated new TSK 0
I20260807 00:40:48.932053 15741 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:40:48.941543 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38967
--builtin_ntp_servers=127.1.116.148:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 00:40:49.036789 15753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:49.037032 15753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:49.037058 15753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:49.038228 15753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:49.038267 15753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:49.038295 15753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:49.039728 15753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:49.039956 15753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:49.040256 15753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:49.045697 15760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:49.045753 15764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:49.045699 15762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:49.046073 15753 server_base.cc:1034] running on GCE node
I20260807 00:40:49.046375 15753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:49.046705 15753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:49.047914 15753 hybrid_clock.cc:648] HybridClock initialized: now 1786063249047891 us; error 43 us; skew 500 ppm
I20260807 00:40:49.049474 15753 webserver.cc:460] Webserver started at http://127.1.116.129:43939/ using document root <none> and password file <none>
I20260807 00:40:49.049808 15753 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:49.049894 15753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:49.050050 15753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:49.051295 15753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "2c7572df69c440a0a4ab9fe4043b63f9"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "4e2b866dc664f9e97a51d4fc074509cb"
server_key_iv: "c193a557d66791df1c6a1d15d0ce1fce"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.051697 15753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "2c7572df69c440a0a4ab9fe4043b63f9"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "4e2b866dc664f9e97a51d4fc074509cb"
server_key_iv: "c193a557d66791df1c6a1d15d0ce1fce"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.053136 15753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 00:40:49.054092 15773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.054354 15753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:49.054476 15753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "2c7572df69c440a0a4ab9fe4043b63f9"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "4e2b866dc664f9e97a51d4fc074509cb"
server_key_iv: "c193a557d66791df1c6a1d15d0ce1fce"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.054627 15753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:49.057972 15753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:49.058241 15753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:49.058391 15753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:49.058661 15753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:49.058995 15753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:49.059060 15753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.059103 15753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:49.059132 15753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.066015 15753 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:41747
I20260807 00:40:49.066135 15928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:41747 every 8 connection(s)
I20260807 00:40:49.066457 15753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:49.068236 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 15753
I20260807 00:40:49.068356 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:49.068567 1490 external_mini_cluster.cc:1428] Setting key 6401ac47ec4ed3c3507bfed62d6f23e1
I20260807 00:40:49.069864 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38967
--builtin_ntp_servers=127.1.116.148:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 00:40:49.070279 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:49.067483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:46797 (local address 127.1.116.190:38967)
0807 00:40:49.067576 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:49.067578 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:49.067816 (+ 238us) server_negotiation.cc:407] Connection header received
0807 00:40:49.068408 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:49.068409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:49.068419 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:49.068477 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:49.070023 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:49.070024 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:49.070025 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:49.070030 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:49.070047 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:49.070052 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:49.070101 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:49.070180 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 00:40:49.070198 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:40:49.070492 15931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:49.067431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38967 (local address 127.1.116.129:46797)
0807 00:40:49.067660 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:49.067705 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:49.068273 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:49.068553 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:49.068558 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:49.068568 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:49.069857 (+ 1289us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:49.069857 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:49.069889 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:49.069919 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:49.070111 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:49.070117 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:49.070120 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:49.070193 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:40:49.070260 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
I20260807 00:40:49.072057 15929 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38967
I20260807 00:40:49.072204 15929 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:49.072371 15929 heartbeater.cc:511] Master 127.1.116.190:38967 requested a full tablet report, sending...
I20260807 00:40:49.073043 15666 ts_manager.cc:194] Registered new tserver with Master: 2c7572df69c440a0a4ab9fe4043b63f9 (127.1.116.129:41747)
W20260807 00:40:49.164585 15932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:49.164842 15932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:49.164866 15932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:49.166002 15932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:49.166041 15932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:49.166067 15932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:49.167479 15932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:49.167714 15932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:49.168018 15932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:49.173650 15938 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:49.173671 15941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:49.173871 15939 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:49.174196 15932 server_base.cc:1034] running on GCE node
I20260807 00:40:49.174454 15932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:49.174831 15932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:49.176020 15932 hybrid_clock.cc:648] HybridClock initialized: now 1786063249176017 us; error 41 us; skew 500 ppm
I20260807 00:40:49.177425 15932 webserver.cc:460] Webserver started at http://127.1.116.130:37185/ using document root <none> and password file <none>
I20260807 00:40:49.177752 15932 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:49.177832 15932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:49.178007 15932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:49.179302 15932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "6d68edb1508f4b0aaa50937e42732cda"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "fbc3c59c8fc962d631b4f7187bb34ec1"
server_key_iv: "22e88832b94bc174b963c3b10c45df38"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.179718 15932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "6d68edb1508f4b0aaa50937e42732cda"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "fbc3c59c8fc962d631b4f7187bb34ec1"
server_key_iv: "22e88832b94bc174b963c3b10c45df38"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.181169 15932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:49.182099 15950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.182356 15932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:49.182478 15932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "6d68edb1508f4b0aaa50937e42732cda"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "fbc3c59c8fc962d631b4f7187bb34ec1"
server_key_iv: "22e88832b94bc174b963c3b10c45df38"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.182590 15932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:49.186213 15932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:49.186506 15932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:49.186697 15932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:49.186944 15932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:49.187287 15932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:49.187348 15932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.187388 15932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:49.187417 15932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.195163 15932 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:39971
I20260807 00:40:49.195335 16105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:39971 every 8 connection(s)
I20260807 00:40:49.195636 15932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:49.197021 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 15932
I20260807 00:40:49.197141 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:49.197394 1490 external_mini_cluster.cc:1428] Setting key d1e9efb6a5e348fc1b9edd32519964eb
I20260807 00:40:49.198783 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38967
--builtin_ntp_servers=127.1.116.148:44651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 00:40:49.199796 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:49.196567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:41003 (local address 127.1.116.190:38967)
0807 00:40:49.196631 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:49.196633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:49.196944 (+ 311us) server_negotiation.cc:407] Connection header received
0807 00:40:49.197603 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:49.197604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:49.197613 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:49.197716 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:49.199428 (+ 1712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:49.199429 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:49.199430 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:49.199435 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:49.199457 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:49.199462 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:49.199532 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:49.199714 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 00:40:49.199731 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 00:40:49.200028 16110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:49.196536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38967 (local address 127.1.116.130:41003)
0807 00:40:49.196768 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:49.196807 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:49.197444 (+ 637us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:49.197804 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:49.197808 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:49.197819 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:49.199195 (+ 1376us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:49.199196 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:49.199238 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:49.199282 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:49.199568 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:49.199574 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:49.199577 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:49.199681 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 00:40:49.199754 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":130,"threads_started":1}
I20260807 00:40:49.200588 16107 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38967
I20260807 00:40:49.200711 16107 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:49.200923 16107 heartbeater.cc:511] Master 127.1.116.190:38967 requested a full tablet report, sending...
I20260807 00:40:49.201462 15666 ts_manager.cc:194] Registered new tserver with Master: 6d68edb1508f4b0aaa50937e42732cda (127.1.116.130:39971)
W20260807 00:40:49.292671 16111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:49.292917 16111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:49.292943 16111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:49.294092 16111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:49.294129 16111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:49.294163 16111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:49.295616 16111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:38967
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:49.295845 16111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:49.296119 16111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:49.301934 16117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:49.301952 16119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:49.302183 16111 server_base.cc:1034] running on GCE node
W20260807 00:40:49.302006 16116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:49.302702 16111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:49.303047 16111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:49.304225 16111 hybrid_clock.cc:648] HybridClock initialized: now 1786063249304185 us; error 66 us; skew 500 ppm
I20260807 00:40:49.305981 16111 webserver.cc:460] Webserver started at http://127.1.116.131:36971/ using document root <none> and password file <none>
I20260807 00:40:49.306264 16111 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:49.306358 16111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:49.306504 16111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:49.307720 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "a6fde520d90448f2888e760762226e84"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "fb00e016e0d1c47b00fccaa2ad0384bf"
server_key_iv: "bdb0dd24dac7c7453a5c1edec3623d06"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.308132 16111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "a6fde520d90448f2888e760762226e84"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "fb00e016e0d1c47b00fccaa2ad0384bf"
server_key_iv: "bdb0dd24dac7c7453a5c1edec3623d06"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.309657 16111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:40:49.310506 16128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.310833 16111 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:49.310951 16111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "a6fde520d90448f2888e760762226e84"
format_stamp: "Formatted at 2026-08-07 00:40:49 on dist-test-slave-h6n0"
server_key: "fb00e016e0d1c47b00fccaa2ad0384bf"
server_key_iv: "bdb0dd24dac7c7453a5c1edec3623d06"
server_key_version: "encryptionkey@0"
I20260807 00:40:49.311046 16111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:49.314865 16111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:49.315165 16111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:49.315317 16111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:49.315580 16111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:49.315928 16111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:49.315994 16111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.316037 16111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:49.316072 16111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:49.322759 16111 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:39485
I20260807 00:40:49.322834 16293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:39485 every 8 connection(s)
I20260807 00:40:49.323117 16111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:49.325956 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 16111
I20260807 00:40:49.326123 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:49.326372 1490 external_mini_cluster.cc:1428] Setting key d12aca3ccafbee512ad6e0888729ae95
I20260807 00:40:49.327075 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:49.324081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:52773 (local address 127.1.116.190:38967)
0807 00:40:49.324195 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:49.324198 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:49.324425 (+ 227us) server_negotiation.cc:407] Connection header received
0807 00:40:49.325062 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:49.325063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:49.325072 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:49.325127 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:49.326711 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:49.326713 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:49.326714 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:40:49.326719 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:49.326737 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:40:49.326742 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:49.326802 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:49.326976 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 00:40:49.326994 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 00:40:49.327309 16296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:49.323978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:38967 (local address 127.1.116.131:52773)
0807 00:40:49.324266 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:49.324307 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:49.324912 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:49.325247 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:49.325251 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:49.325261 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:49.326487 (+ 1226us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:40:49.326488 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:49.326573 (+ 85us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:40:49.326616 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:49.326883 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:49.326889 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:49.326892 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:40:49.326988 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 00:40:49.327060 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":107,"threads_started":1}
I20260807 00:40:49.328229 16294 heartbeater.cc:347] Connected to a master server at 127.1.116.190:38967
I20260807 00:40:49.328344 16294 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:49.328475 16294 heartbeater.cc:511] Master 127.1.116.190:38967 requested a full tablet report, sending...
I20260807 00:40:49.328902 15666 ts_manager.cc:194] Registered new tserver with Master: a6fde520d90448f2888e760762226e84 (127.1.116.131:39485)
I20260807 00:40:49.339355 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 00:40:49.341790 15725 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:40:49.340988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38416 (local address 127.1.116.190:38967)
0807 00:40:49.341104 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:49.341107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:49.341347 (+ 240us) server_negotiation.cc:407] Connection header received
0807 00:40:49.341378 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:49.341379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:49.341384 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:49.341435 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:49.341663 (+ 228us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38416: BlockingRecv error: recv got EOF from 127.0.0.1:38416 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 00:40:49.341804 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 15753
I20260807 00:40:49.359489 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 15932
I20260807 00:40:49.367434 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 16111
I20260807 00:40:49.376201 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 15627
2026-08-07T00:40:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (642 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:49 dist-test-slave-h6n0 krb5kdc[16305](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:49 dist-test-slave-h6n0 krb5kdc[16305](info): set up 2 sockets
Aug 07 00:40:49 dist-test-slave-h6n0 krb5kdc[16305](info): commencing operation
krb5kdc: starting...
W20260807 00:40:50.647301 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.246s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:50 dist-test-slave-h6n0 krb5kdc[16305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:40:50.779568 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:33023
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:33023
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:50.873090 16322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:50.873337 16322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:50.873358 16322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:50.874588 16322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:40:50.874643 16322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:50.874660 16322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:50.874672 16322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:40:50.874684 16322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:40:50.876230 16322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:33023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:33023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:50.876504 16322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:50.876775 16322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:50.882265 16327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:50.882283 16331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:50.882283 16335 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:50.882779 16322 server_base.cc:1034] running on GCE node
I20260807 00:40:50.883072 16322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:50.883373 16322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:50.884565 16322 hybrid_clock.cc:648] HybridClock initialized: now 1786063250884537 us; error 55 us; skew 500 ppm
Aug 07 00:40:50 dist-test-slave-h6n0 krb5kdc[16305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063250, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:50.887382 16322 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:40:50.887910 16322 webserver.cc:460] Webserver started at http://127.1.116.190:35559/ using document root <none> and password file <none>
I20260807 00:40:50.888131 16322 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:50.888181 16322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:50.888296 16322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:50.889401 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "bee2ed18715349168bc3561b643dcadb"
format_stamp: "Formatted at 2026-08-07 00:40:50 on dist-test-slave-h6n0"
server_key: "57123099b0e5d11d35315d3a6e756a84"
server_key_iv: "d9bc04706559109f518bf9bded96f911"
server_key_version: "encryptionkey@0"
I20260807 00:40:50.889755 16322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "bee2ed18715349168bc3561b643dcadb"
format_stamp: "Formatted at 2026-08-07 00:40:50 on dist-test-slave-h6n0"
server_key: "57123099b0e5d11d35315d3a6e756a84"
server_key_iv: "d9bc04706559109f518bf9bded96f911"
server_key_version: "encryptionkey@0"
I20260807 00:40:50.891340 16322 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:50.892158 16347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:50.892444 16322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:50.892568 16322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "bee2ed18715349168bc3561b643dcadb"
format_stamp: "Formatted at 2026-08-07 00:40:50 on dist-test-slave-h6n0"
server_key: "57123099b0e5d11d35315d3a6e756a84"
server_key_iv: "d9bc04706559109f518bf9bded96f911"
server_key_version: "encryptionkey@0"
I20260807 00:40:50.892663 16322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:50.907902 16322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:50.910373 16322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:50.910651 16322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:50.915637 16322 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:33023
I20260807 00:40:50.915712 16422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:33023 every 8 connection(s)
I20260807 00:40:50.916081 16322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:40:50.917198 16423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:50.919678 16423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb: Bootstrap starting.
I20260807 00:40:50.920250 16423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:50.920491 16423 log.cc:824] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:50.921289 16423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb: No bootstrap required, opened a new log
I20260807 00:40:50.922971 16423 raft_consensus.cc:348] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee2ed18715349168bc3561b643dcadb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33023 } }
I20260807 00:40:50.923115 16423 raft_consensus.cc:374] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:50.923137 16423 raft_consensus.cc:729] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bee2ed18715349168bc3561b643dcadb, State: Initialized, Role: FOLLOWER
I20260807 00:40:50.923273 16423 consensus_queue.cc:260] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee2ed18715349168bc3561b643dcadb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33023 } }
I20260807 00:40:50.923357 16423 raft_consensus.cc:388] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:40:50.923379 16423 raft_consensus.cc:482] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:40:50.923408 16423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:50.924167 16423 raft_consensus.cc:504] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee2ed18715349168bc3561b643dcadb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33023 } }
I20260807 00:40:50.924259 16423 leader_election.cc:302] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bee2ed18715349168bc3561b643dcadb; no voters:
I20260807 00:40:50.924412 16423 leader_election.cc:288] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:40:50.924439 16429 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:50.924563 16429 raft_consensus.cc:686] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [term 1 LEADER]: Becoming Leader. State: Replica: bee2ed18715349168bc3561b643dcadb, State: Running, Role: LEADER
I20260807 00:40:50.924654 16429 consensus_queue.cc:237] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee2ed18715349168bc3561b643dcadb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33023 } }
I20260807 00:40:50.924697 16423 sys_catalog.cc:564] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:40:50.925040 16430 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [sys.catalog]: SysCatalogTable state changed. Reason: New leader bee2ed18715349168bc3561b643dcadb. Latest consensus state: current_term: 1 leader_uuid: "bee2ed18715349168bc3561b643dcadb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee2ed18715349168bc3561b643dcadb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33023 } } }
I20260807 00:40:50.925084 16431 sys_catalog.cc:455] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bee2ed18715349168bc3561b643dcadb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bee2ed18715349168bc3561b643dcadb" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33023 } } }
I20260807 00:40:50.925379 16431 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:50.925635 16430 sys_catalog.cc:458] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb [sys.catalog]: This master's current role is: LEADER
I20260807 00:40:50.925688 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 16322
I20260807 00:40:50.925809 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:40:50.925989 16439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:40:50.926069 1490 external_mini_cluster.cc:1428] Setting key 7d381ab39acffb371f1b7710445f40ae
I20260807 00:40:50.926714 16439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:40:50.928594 16439 catalog_manager.cc:1347] Generated new cluster ID: ad2065b007924cd9a8e662939ed5f8a4
I20260807 00:40:50.928653 16439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:40:50 dist-test-slave-h6n0 krb5kdc[16305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063250, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:50.942271 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:50.926650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38700 (local address 127.1.116.190:33023)
0807 00:40:50.930718 (+ 4068us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:50.930729 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:50.930770 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:40:50.930907 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:50.930922 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:50.930986 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:50.931153 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:50.931992 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:50.933510 (+ 1518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:50.934675 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:50.935326 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:50.935432 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:50.938444 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:50.938449 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:50.938450 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:50.938464 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:50.940520 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:50.940954 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:50.940957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:50.940957 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:50.941065 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:50.941297 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:50.941298 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:50.941298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:50.941624 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:50.941764 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:50.941901 (+ 137us) server_negotiation.cc:299] Negotiation successful
0807 00:40:50.942045 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3994,"thread_start_us":84,"threads_started":1}
I20260807 00:40:50.945295 16439 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:40:50.945822 16439 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:40:50.950330 16439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb: Generated new TSK 0
I20260807 00:40:50.950567 16439 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:40:50.983304 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33023
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:51.079691 16457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:51.079929 16457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:51.079955 16457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:51.081164 16457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:51.081223 16457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:51.081251 16457 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:40:51.082847 16457 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:51.083122 16457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:51.083398 16457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:51.089104 16462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:51.089311 16457 server_base.cc:1034] running on GCE node
W20260807 00:40:51.089365 16467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:51.089124 16463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:51.089689 16457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:51.089983 16457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:51.091212 16457 hybrid_clock.cc:648] HybridClock initialized: now 1786063251091197 us; error 28 us; skew 500 ppm
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[16305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:51.094100 16457 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:40:51.094688 16457 webserver.cc:460] Webserver started at http://127.1.116.129:33641/ using document root <none> and password file <none>
I20260807 00:40:51.094895 16457 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:51.094942 16457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:51.095090 16457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:51.096306 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "1166def5f4a249e9ad500d6049c73b44"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "52c48cef058af2013d5188bb839ecbe1"
server_key_iv: "8f13b1769afb9b52436046c294f96b2a"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.096644 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "1166def5f4a249e9ad500d6049c73b44"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "52c48cef058af2013d5188bb839ecbe1"
server_key_iv: "8f13b1769afb9b52436046c294f96b2a"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.098210 16457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:40:51.099100 16480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.099411 16457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:40:51.099541 16457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "1166def5f4a249e9ad500d6049c73b44"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "52c48cef058af2013d5188bb839ecbe1"
server_key_iv: "8f13b1769afb9b52436046c294f96b2a"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.099632 16457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:51.115280 16457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:51.117831 16457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:51.118003 16457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:51.118283 16457 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:51.118690 16457 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:51.118747 16457 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.118784 16457 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:51.118825 16457 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.126891 16457 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:37631
I20260807 00:40:51.126948 16650 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:37631 every 8 connection(s)
I20260807 00:40:51.127339 16457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:40:51.129770 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 16457
I20260807 00:40:51.129864 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:40:51.130110 1490 external_mini_cluster.cc:1428] Setting key 78eea6c52fa0d82b177ba291a9b4e1cb
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[16305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:51.138206 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.128244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:48253 (local address 127.1.116.190:33023)
0807 00:40:51.128315 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.128318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.128597 (+ 279us) server_negotiation.cc:407] Connection header received
0807 00:40:51.129192 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.129193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.129212 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.129284 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:51.130648 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.131187 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.132208 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.132439 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.132537 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.135204 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:51.135205 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:51.135206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:51.135214 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:51.136740 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.137176 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.137178 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.137178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.137242 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.137505 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.137506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.137506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.137625 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:51.137705 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.137981 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.138097 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:40:51.138451 16654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.128222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33023 (local address 127.1.116.129:48253)
0807 00:40:51.128425 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.128474 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.129029 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.129429 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.129433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.130255 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:51.130478 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.130483 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.131292 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.131293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.132105 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.132108 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.132550 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.132551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.132680 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.133286 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:51.133295 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:51.135100 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:51.136846 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:51.136854 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:51.136857 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:51.137097 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:51.137314 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:51.137315 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:51.137315 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:51.137434 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:51.137697 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:51.137704 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:51.137843 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.137955 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.138108 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":110,"threads_started":1}
I20260807 00:40:51.139215 16652 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33023
I20260807 00:40:51.139317 16652 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:51.139520 16652 heartbeater.cc:511] Master 127.1.116.190:33023 requested a full tablet report, sending...
I20260807 00:40:51.140111 16368 ts_manager.cc:194] Registered new tserver with Master: 1166def5f4a249e9ad500d6049c73b44 (127.1.116.129:37631)
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:40:51.140772 16368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:48253
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:40:51.160838 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33023
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:51.253453 16657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:51.253692 16657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:51.253716 16657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:51.254840 16657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:51.254879 16657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:51.254905 16657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:40:51.256348 16657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:51.256572 16657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:51.256844 16657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:51.262863 16667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:51.262868 16664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:51.263029 16657 server_base.cc:1034] running on GCE node
W20260807 00:40:51.263185 16662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:51.263459 16657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:51.263770 16657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:51.264958 16657 hybrid_clock.cc:648] HybridClock initialized: now 1786063251264912 us; error 58 us; skew 500 ppm
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[16305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:51.268033 16657 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:40:51.268628 16657 webserver.cc:460] Webserver started at http://127.1.116.130:40331/ using document root <none> and password file <none>
I20260807 00:40:51.268913 16657 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:51.268985 16657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:51.269145 16657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:51.270358 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "d648e2d691194f609df60dab9c665d16"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "6d4a6396ea9e80f3a1e71e77c65fe49a"
server_key_iv: "cf2f19587ba7d7cc1ed48429c227792e"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.270792 16657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "d648e2d691194f609df60dab9c665d16"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "6d4a6396ea9e80f3a1e71e77c65fe49a"
server_key_iv: "cf2f19587ba7d7cc1ed48429c227792e"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.272356 16657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:40:51.273325 16674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.273586 16657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:40:51.273706 16657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "d648e2d691194f609df60dab9c665d16"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "6d4a6396ea9e80f3a1e71e77c65fe49a"
server_key_iv: "cf2f19587ba7d7cc1ed48429c227792e"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.273785 16657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:51.292536 16657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:51.295100 16657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:51.295388 16657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:51.295732 16657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:51.296131 16657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:51.296190 16657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.296249 16657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:51.296278 16657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.304066 16657 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:38495
I20260807 00:40:51.304206 16821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:38495 every 8 connection(s)
I20260807 00:40:51.304608 16657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:40:51.307832 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 16657
I20260807 00:40:51.308000 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:40:51.308343 1490 external_mini_cluster.cc:1428] Setting key 476049bcc0b4aad98bcd345dec75ceb0
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[16305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:51.317327 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.305506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:36381 (local address 127.1.116.190:33023)
0807 00:40:51.305596 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.305599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.305927 (+ 328us) server_negotiation.cc:407] Connection header received
0807 00:40:51.306509 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.306511 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.306783 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.307009 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:51.308239 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.309096 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.310354 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.310739 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.310848 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.313657 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:51.313659 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:51.313659 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:51.313668 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:51.315566 (+ 1898us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.316054 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.316055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.316056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.316125 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.316525 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.316526 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.316526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.316692 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:51.316767 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.317097 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.317201 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:40:51.317431 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.305500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33023 (local address 127.1.116.130:36381)
0807 00:40:51.305738 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.305780 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.306375 (+ 595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.307117 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.307122 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.307792 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:51.308100 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.308107 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.309329 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.309330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.310213 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.310215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.310851 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.310851 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.310979 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.311781 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:51.311792 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:51.313542 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:51.315675 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:51.315683 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:51.315685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:51.315953 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:51.316256 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:51.316257 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:51.316258 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:51.316427 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:51.316794 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:51.316800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:51.316938 (+ 138us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.317033 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.317187 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":124,"threads_started":1}
I20260807 00:40:51.318087 16822 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33023
I20260807 00:40:51.318192 16822 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:51.318439 16822 heartbeater.cc:511] Master 127.1.116.190:33023 requested a full tablet report, sending...
I20260807 00:40:51.319000 16368 ts_manager.cc:194] Registered new tserver with Master: d648e2d691194f609df60dab9c665d16 (127.1.116.130:38495)
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
I20260807 00:40:51.319484 16368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:36381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:40:51.338021 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33023
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:40233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:40:51.423903 16830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:40:51.424141 16830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:40:51.424180 16830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:40:51.425596 16830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:40:51.425666 16830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:40:51.425715 16830 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:40:51.427500 16830 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33023
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:40:51.427767 16830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:40:51.428004 16830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:40:51.433894 16845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:51.434000 16830 server_base.cc:1034] running on GCE node
W20260807 00:40:51.433876 16843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:40:51.433919 16842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:40:51.434446 16830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:40:51.434832 16830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:40:51.436003 16830 hybrid_clock.cc:648] HybridClock initialized: now 1786063251435992 us; error 41 us; skew 500 ppm
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[16305](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:40:51.439275 16830 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:40:51.439783 16830 webserver.cc:460] Webserver started at http://127.1.116.131:39691/ using document root <none> and password file <none>
I20260807 00:40:51.440002 16830 fs_manager.cc:359] Metadata directory not provided
I20260807 00:40:51.440052 16830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:40:51.440147 16830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:40:51.441138 16830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "1cb49b1da8b943e3862348dc40251909"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "c13fb922c6cad867c012324f94a9b3e0"
server_key_iv: "8e9204bec071b53d54b149a1d419c0cf"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.441448 16830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "1cb49b1da8b943e3862348dc40251909"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "c13fb922c6cad867c012324f94a9b3e0"
server_key_iv: "8e9204bec071b53d54b149a1d419c0cf"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.442893 16830 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:40:51.443908 16855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.444352 16830 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:40:51.444502 16830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "1cb49b1da8b943e3862348dc40251909"
format_stamp: "Formatted at 2026-08-07 00:40:51 on dist-test-slave-h6n0"
server_key: "c13fb922c6cad867c012324f94a9b3e0"
server_key_iv: "8e9204bec071b53d54b149a1d419c0cf"
server_key_version: "encryptionkey@0"
I20260807 00:40:51.444595 16830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:40:51.460228 16830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:40:51.462484 16830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:40:51.462693 16830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:40:51.462883 16830 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:40:51.463212 16830 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:40:51.463274 16830 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.463310 16830 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:40:51.463357 16830 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:40:51.470307 16830 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:43173
I20260807 00:40:51.470400 17005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:43173 every 8 connection(s)
I20260807 00:40:51.470744 16830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:40:51.474525 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 16830
I20260807 00:40:51.474680 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:40:51.474902 1490 external_mini_cluster.cc:1428] Setting key eb159308ece0f24dea381865be8399ca
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[16305](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063251, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:40:51.481474 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.471726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:36439 (local address 127.1.116.190:33023)
0807 00:40:51.471834 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.471838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.471990 (+ 152us) server_negotiation.cc:407] Connection header received
0807 00:40:51.472602 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.472603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.472615 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.472680 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:51.473784 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.474332 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.475977 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.476212 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.476275 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.478688 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:51.478690 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:51.478690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:51.478698 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:51.480045 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.480544 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.480544 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.480545 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.480605 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.480889 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.480889 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.480889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.481002 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:51.481060 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.481325 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.481398 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 00:40:51.481662 17010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.471640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33023 (local address 127.1.116.131:36439)
0807 00:40:51.471824 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.471868 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.472501 (+ 633us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.472748 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.472752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.473404 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:40:51.473637 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.473643 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.474445 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.474446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.475871 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.475874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.476296 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.476297 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.476426 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.476926 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:40:51.476953 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:40:51.478503 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:40:51.480161 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:51.480169 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:51.480170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:51.480457 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:51.480696 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:40:51.480697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:40:51.480697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:40:51.480812 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:40:51.481071 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:40:51.481076 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:40:51.481199 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.481289 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.481432 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260807 00:40:51.482249 17007 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33023
I20260807 00:40:51.482376 17007 heartbeater.cc:464] Registering TS with master...
I20260807 00:40:51.482643 17007 heartbeater.cc:511] Master 127.1.116.190:33023 requested a full tablet report, sending...
I20260807 00:40:51.483098 16368 ts_manager.cc:194] Registered new tserver with Master: 1cb49b1da8b943e3862348dc40251909 (127.1.116.131:43173)
I20260807 00:40:51.483494 16368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:36439
I20260807 00:40:51.486958 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:40:51.495468 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.488399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.1.116.190:33023)
0807 00:40:51.488490 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.488492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.488565 (+ 73us) server_negotiation.cc:407] Connection header received
0807 00:40:51.488608 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.488609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.488618 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.488703 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:40:51.489315 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.489779 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.490817 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.491010 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.491075 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.492324 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:40:51.492325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:40:51.492325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:40:51.492332 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:40:51.494180 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.494606 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.494607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.494608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.494665 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:40:51.494898 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:40:51.494899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:40:51.494900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:40:51.495050 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:40:51.495151 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.495297 (+ 146us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.495364 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:40:51.497372 16368 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:40:51.499467 16368 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:40:51.507741 17021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.502929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37631 (local address 127.0.0.1:60696)
0807 00:40:51.503940 (+ 1011us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.503952 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.504021 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.504264 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.504265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.504266 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:51.504322 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.504324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.505637 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.505639 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.506638 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.506646 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.507462 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.507464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.507535 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.507548 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.507616 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.507673 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":987,"mutex_wait_us":5,"spinlock_wait_cycles":1280,"thread_start_us":89,"threads_started":1}
I20260807 00:40:51.508024 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.503217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60696 (local address 127.1.116.129:37631)
0807 00:40:51.503494 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.503499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.504080 (+ 581us) server_negotiation.cc:407] Connection header received
0807 00:40:51.504136 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.504139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.504159 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.504262 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:51.504393 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.505417 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.506741 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.507362 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.507441 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.507495 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.507660 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.507701 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":115,"threads_started":1}
I20260807 00:40:51.508831 16556 tablet_service.cc:1467] Processing CreateTablet for tablet afef1435eac14f65acc3a4c8ebed6f08 (DEFAULT_TABLE table=test-table [id=1d3005fc20864b58a5445338a3de64d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:51.508996 17019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.502930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:43173 (local address 127.0.0.1:39956)
0807 00:40:51.503282 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.503332 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.503464 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.503872 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.503876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.503879 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:51.503972 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.503975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.505397 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.505398 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.506953 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.506956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.508725 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.508727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.508795 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.508801 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.508877 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.508932 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":94,"thread_start_us":74,"threads_started":1}
I20260807 00:40:51.509042 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.503186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39956 (local address 127.1.116.131:43173)
0807 00:40:51.503623 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.503628 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.503650 (+ 22us) server_negotiation.cc:407] Connection header received
0807 00:40:51.503710 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.503712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.503727 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.503824 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:51.504106 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.505279 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.508006 (+ 2727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.508631 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.508695 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.508728 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.508918 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.508961 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":136,"threads_started":1}
I20260807 00:40:51.509195 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.503306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.1.116.130:38495)
0807 00:40:51.503644 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.503648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.504181 (+ 533us) server_negotiation.cc:407] Connection header received
0807 00:40:51.504407 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.504409 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.504423 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.504511 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:51.504630 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.505413 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.507075 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.507831 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.507911 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.507968 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.509051 (+ 1083us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.509103 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":105,"threads_started":1}
I20260807 00:40:51.509394 16556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afef1435eac14f65acc3a4c8ebed6f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:51.509646 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.502970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:38495 (local address 127.0.0.1:60806)
0807 00:40:51.504100 (+ 1130us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.504116 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.504179 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.504520 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.504521 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.504524 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:51.504566 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.504567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.505560 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.505561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.506953 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.506956 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.508767 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.508768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.508829 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.508835 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.508912 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.508981 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1112,"mutex_wait_us":80,"thread_start_us":102,"threads_started":1}
I20260807 00:40:51.510655 16740 tablet_service.cc:1467] Processing CreateTablet for tablet afef1435eac14f65acc3a4c8ebed6f08 (DEFAULT_TABLE table=test-table [id=1d3005fc20864b58a5445338a3de64d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:51.511034 16922 tablet_service.cc:1467] Processing CreateTablet for tablet afef1435eac14f65acc3a4c8ebed6f08 (DEFAULT_TABLE table=test-table [id=1d3005fc20864b58a5445338a3de64d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:40:51.511287 16740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afef1435eac14f65acc3a4c8ebed6f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:51.511384 16922 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet afef1435eac14f65acc3a4c8ebed6f08. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:40:51.513612 17028 tablet_bootstrap.cc:492] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44: Bootstrap starting.
I20260807 00:40:51.514770 17028 tablet_bootstrap.cc:654] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:51.514968 17029 tablet_bootstrap.cc:492] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Bootstrap starting.
I20260807 00:40:51.515586 17029 tablet_bootstrap.cc:654] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:51.515779 17028 log.cc:824] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:51.516185 17032 tablet_bootstrap.cc:492] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Bootstrap starting.
I20260807 00:40:51.516290 17029 log.cc:824] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:51.516862 17032 tablet_bootstrap.cc:654] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Neither blocks nor log segments found. Creating new log.
I20260807 00:40:51.516917 17028 tablet_bootstrap.cc:492] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44: No bootstrap required, opened a new log
I20260807 00:40:51.516985 17028 ts_tablet_manager.cc:1397] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:40:51.517434 17032 log.cc:824] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Log is configured to *not* fsync() on all Append() calls
I20260807 00:40:51.518229 17032 tablet_bootstrap.cc:492] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: No bootstrap required, opened a new log
I20260807 00:40:51.518306 17032 ts_tablet_manager.cc:1397] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:40:51.519279 17028 raft_consensus.cc:348] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.519469 17028 raft_consensus.cc:374] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:51.519508 17028 raft_consensus.cc:729] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1166def5f4a249e9ad500d6049c73b44, State: Initialized, Role: FOLLOWER
I20260807 00:40:51.519641 17028 consensus_queue.cc:260] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.519941 17028 ts_tablet_manager.cc:1428] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:40:51.519981 16652 heartbeater.cc:503] Master 127.1.116.190:33023 was elected leader, sending a full tablet report...
I20260807 00:40:51.520653 17029 tablet_bootstrap.cc:492] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: No bootstrap required, opened a new log
I20260807 00:40:51.520737 17029 ts_tablet_manager.cc:1397] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 00:40:51.520658 17032 raft_consensus.cc:348] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.520795 17032 raft_consensus.cc:374] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:51.520978 17032 raft_consensus.cc:729] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d648e2d691194f609df60dab9c665d16, State: Initialized, Role: FOLLOWER
I20260807 00:40:51.521128 17032 consensus_queue.cc:260] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.521441 17032 ts_tablet_manager.cc:1428] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:40:51.521605 16822 heartbeater.cc:503] Master 127.1.116.190:33023 was elected leader, sending a full tablet report...
I20260807 00:40:51.522487 17029 raft_consensus.cc:348] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.522686 17029 raft_consensus.cc:374] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:40:51.522723 17029 raft_consensus.cc:729] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cb49b1da8b943e3862348dc40251909, State: Initialized, Role: FOLLOWER
I20260807 00:40:51.522807 17029 consensus_queue.cc:260] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.523006 17029 ts_tablet_manager.cc:1428] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260807 00:40:51.523087 17007 heartbeater.cc:503] Master 127.1.116.190:33023 was elected leader, sending a full tablet report...
W20260807 00:40:51.555451 16823 tablet.cc:2369] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:40:51.628299 16653 tablet.cc:2369] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:40:51.640280 17036 raft_consensus.cc:482] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:40:51.640424 17036 raft_consensus.cc:504] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.640799 17036 leader_election.cc:288] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cb49b1da8b943e3862348dc40251909 (127.1.116.131:43173), d648e2d691194f609df60dab9c665d16 (127.1.116.130:38495)
I20260807 00:40:51.646014 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.641180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:36087 (local address 127.1.116.131:43173)
0807 00:40:51.641247 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.641249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.641458 (+ 209us) server_negotiation.cc:407] Connection header received
0807 00:40:51.641539 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.641540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.641550 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.641646 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:51.641988 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.642906 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.644774 (+ 1868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.645548 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.645643 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.645653 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.645888 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.645941 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:40:51.646044 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.641064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:38495 (local address 127.1.116.129:53277)
0807 00:40:51.641335 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.641350 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.641443 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.641707 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.641709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.641714 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:51.641882 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.641885 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.643031 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.643033 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.644536 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.644539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.645678 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.645680 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.645776 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.645785 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.645860 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.645924 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":87,"spinlock_wait_cycles":17536,"thread_start_us":168,"threads_started":1}
I20260807 00:40:51.646080 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.641266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:53277 (local address 127.1.116.130:38495)
0807 00:40:51.641312 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.641314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.641438 (+ 124us) server_negotiation.cc:407] Connection header received
0807 00:40:51.641537 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.641538 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.641550 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.641646 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:40:51.641988 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.642907 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.644767 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.645540 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.645643 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.645653 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.645928 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.645976 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:40:51.646353 17039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.641063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:43173 (local address 127.1.116.129:36087)
0807 00:40:51.641335 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.641350 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:40:51.641443 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:40:51.641707 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:40:51.641709 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:40:51.641714 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:40:51.641878 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.641881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.643033 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.643033 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.644536 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:40:51.644539 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.645680 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.645681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:40:51.645776 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.645786 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 00:40:51.645857 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:40:51.646005 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":120,"thread_start_us":147,"threads_started":1}
I20260807 00:40:51.646669 16764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afef1435eac14f65acc3a4c8ebed6f08" candidate_uuid: "1166def5f4a249e9ad500d6049c73b44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d648e2d691194f609df60dab9c665d16" is_pre_election: true
I20260807 00:40:51.646788 16946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afef1435eac14f65acc3a4c8ebed6f08" candidate_uuid: "1166def5f4a249e9ad500d6049c73b44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb49b1da8b943e3862348dc40251909" is_pre_election: true
I20260807 00:40:51.646916 16764 raft_consensus.cc:2454] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1166def5f4a249e9ad500d6049c73b44 in term 0.
I20260807 00:40:51.646948 16946 raft_consensus.cc:2454] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1166def5f4a249e9ad500d6049c73b44 in term 0.
I20260807 00:40:51.647238 16483 leader_election.cc:302] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1166def5f4a249e9ad500d6049c73b44, d648e2d691194f609df60dab9c665d16; no voters:
I20260807 00:40:51.647480 17036 raft_consensus.cc:2781] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:40:51.647631 17036 raft_consensus.cc:482] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:40:51.647662 17036 raft_consensus.cc:3037] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:51.648761 17036 raft_consensus.cc:504] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.648933 17036 leader_election.cc:288] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [CANDIDATE]: Term 1 election: Requested vote from peers 1cb49b1da8b943e3862348dc40251909 (127.1.116.131:43173), d648e2d691194f609df60dab9c665d16 (127.1.116.130:38495)
I20260807 00:40:51.649106 16764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afef1435eac14f65acc3a4c8ebed6f08" candidate_uuid: "1166def5f4a249e9ad500d6049c73b44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d648e2d691194f609df60dab9c665d16"
I20260807 00:40:51.649138 16946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "afef1435eac14f65acc3a4c8ebed6f08" candidate_uuid: "1166def5f4a249e9ad500d6049c73b44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cb49b1da8b943e3862348dc40251909"
I20260807 00:40:51.649215 16946 raft_consensus.cc:3037] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:51.649221 16764 raft_consensus.cc:3037] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:40:51.649940 16764 raft_consensus.cc:2454] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1166def5f4a249e9ad500d6049c73b44 in term 1.
I20260807 00:40:51.650163 16483 leader_election.cc:302] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1166def5f4a249e9ad500d6049c73b44, d648e2d691194f609df60dab9c665d16; no voters:
I20260807 00:40:51.650208 16946 raft_consensus.cc:2454] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1166def5f4a249e9ad500d6049c73b44 in term 1.
I20260807 00:40:51.650267 17036 raft_consensus.cc:2781] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:40:51.650472 17036 raft_consensus.cc:686] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [term 1 LEADER]: Becoming Leader. State: Replica: 1166def5f4a249e9ad500d6049c73b44, State: Running, Role: LEADER
I20260807 00:40:51.650741 17036 consensus_queue.cc:237] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } }
I20260807 00:40:51.651649 16367 catalog_manager.cc:5495] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1166def5f4a249e9ad500d6049c73b44 (127.1.116.129). New cstate: current_term: 1 leader_uuid: "1166def5f4a249e9ad500d6049c73b44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1166def5f4a249e9ad500d6049c73b44" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 } health_report { overall_health: UNKNOWN } } }
I20260807 00:40:51.690413 17022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.687155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60698 (local address 127.1.116.129:37631)
0807 00:40:51.687290 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 00:40:51.687292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:40:51.687305 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:40:51.687354 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:40:51.687355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:40:51.687379 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:40:51.687547 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:40:51.687776 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.688385 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.689377 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:40:51.689599 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:40:51.689665 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:40:51.689837 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:40:51.689967 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:40:51.690052 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 00:40:51.690236 (+ 184us) server_negotiation.cc:299] Negotiation successful
0807 00:40:51.690260 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260807 00:40:51.692765 16946 raft_consensus.cc:1261] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 1 FOLLOWER]: Refusing update from remote peer 1166def5f4a249e9ad500d6049c73b44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:51.692765 16764 raft_consensus.cc:1261] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 1 FOLLOWER]: Refusing update from remote peer 1166def5f4a249e9ad500d6049c73b44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:40:51.693320 17041 consensus_queue.cc:1035] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cb49b1da8b943e3862348dc40251909" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 43173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:51.693513 17036 consensus_queue.cc:1035] T afef1435eac14f65acc3a4c8ebed6f08 P 1166def5f4a249e9ad500d6049c73b44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d648e2d691194f609df60dab9c665d16" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 38495 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:40:51.696795 17045 mvcc.cc:205] Tried to move back new op lower bound from 7315715078930354176 to 7315715078762061824. Current Snapshot: MvccSnapshot[applied={T|T < 7315715078930354176}]
I20260807 00:40:51.696936 17048 mvcc.cc:205] Tried to move back new op lower bound from 7315715078930354176 to 7315715078762061824. Current Snapshot: MvccSnapshot[applied={T|T < 7315715078930354176}]
I20260807 00:40:51.697481 17046 mvcc.cc:205] Tried to move back new op lower bound from 7315715078930354176 to 7315715078762061824. Current Snapshot: MvccSnapshot[applied={T|T < 7315715078930354176}]
I20260807 00:40:51.699759 16367 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:51.699868 16367 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:40:51.700850 16367 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bee2ed18715349168bc3561b643dcadb: Sending DeleteTablet for 3 replicas of tablet afef1435eac14f65acc3a4c8ebed6f08
I20260807 00:40:51.701303 16740 tablet_service.cc:1514] Processing DeleteTablet for tablet afef1435eac14f65acc3a4c8ebed6f08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:51 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:60806
I20260807 00:40:51.701601 17059 tablet_replica.cc:330] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: stopping tablet replica
I20260807 00:40:51.701603 16922 tablet_service.cc:1514] Processing DeleteTablet for tablet afef1435eac14f65acc3a4c8ebed6f08 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:40:51 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:39956
I20260807 00:40:51.701694 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 16457
I20260807 00:40:51.701713 17059 raft_consensus.cc:2229] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:51.701848 17059 raft_consensus.cc:2258] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:51.702397 17060 tablet_replica.cc:330] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: stopping tablet replica
I20260807 00:40:51.702500 17060 raft_consensus.cc:2229] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:40:51.702658 17060 raft_consensus.cc:2258] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:40:51.702867 17059 ts_tablet_manager.cc:1905] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:51.704753 17060 ts_tablet_manager.cc:1905] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:40:51.704936 17059 ts_tablet_manager.cc:1918] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:51.705009 17059 log.cc:1196] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/afef1435eac14f65acc3a4c8ebed6f08
I20260807 00:40:51.705363 17059 ts_tablet_manager.cc:1939] T afef1435eac14f65acc3a4c8ebed6f08 P d648e2d691194f609df60dab9c665d16: Deleting consensus metadata
I20260807 00:40:51.705900 16350 catalog_manager.cc:4841] TS d648e2d691194f609df60dab9c665d16 (127.1.116.130:38495): tablet afef1435eac14f65acc3a4c8ebed6f08 (table test-table [id=1d3005fc20864b58a5445338a3de64d8]) successfully deleted
I20260807 00:40:51.706664 17060 ts_tablet_manager.cc:1918] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:40:51.706748 17060 log.cc:1196] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/afef1435eac14f65acc3a4c8ebed6f08
I20260807 00:40:51.707046 17060 ts_tablet_manager.cc:1939] T afef1435eac14f65acc3a4c8ebed6f08 P 1cb49b1da8b943e3862348dc40251909: Deleting consensus metadata
I20260807 00:40:51.707437 16349 catalog_manager.cc:4841] TS 1cb49b1da8b943e3862348dc40251909 (127.1.116.131:43173): tablet afef1435eac14f65acc3a4c8ebed6f08 (table test-table [id=1d3005fc20864b58a5445338a3de64d8]) successfully deleted
W20260807 00:40:51.715540 16348 connection.cc:535] client connection to 127.1.116.129:37631 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 00:40:51.715664 16348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:40:51.715866 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 16657
I20260807 00:40:51.719177 17020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:40:51.716035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37631 (local address 127.0.0.1:60710)
0807 00:40:51.719088 (+ 3053us) negotiation.cc:107] Waiting for socket to connect
0807 00:40:51.719134 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37631: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3019}
W20260807 00:40:51.721421 16348 catalog_manager.cc:4584] TS 1166def5f4a249e9ad500d6049c73b44 (127.1.116.129:37631): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet afef1435eac14f65acc3a4c8ebed6f08: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37631: connect: Connection refused (error 111)
I20260807 00:40:51.726874 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 16830
I20260807 00:40:51.740672 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 16322
2026-08-07T00:40:51Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2366 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[17067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[17067](info): set up 2 sockets
Aug 07 00:40:51 dist-test-slave-h6n0 krb5kdc[17067](info): commencing operation
krb5kdc: starting...
W20260807 00:40:52.999505 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:40:53 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:40:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:40:53Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/rangeradmin_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/rangeradmin_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/rangerlookup_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/rangerlookup_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
I20260807 00:40:53.183585 1490 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLvHwjp/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/postgres -l logfile start
2026-08-07 00:40:55.719 UTC [17108] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 00:40:55.719 UTC [17108] LOG: listening on IPv4 address "127.1.116.148", port 57265
2026-08-07 00:40:55.722 UTC [17108] LOG: listening on Unix socket "/tmp/.s.PGSQL.57265"
2026-08-07 00:40:55.757 UTC [17111] LOG: database system was shut down at 2026-08-07 00:40:54 UTC
2026-08-07 00:40:55.761 UTC [17108] LOG: database system is ready to accept connections
I20260807 00:40:56.941874 1490 mini_postgres.cc:96] Postgres bound to 57265
2026-08-07 00:40:56.950 UTC [17122] FATAL: database "slave" does not exist
127.1.116.148:57265 - accepting connections
I20260807 00:40:56.951280 1490 mini_ranger.cc:161] Starting Ranger...
I20260807 00:40:56.972195 1490 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 00:40:57.434947 1490 mini_ranger.cc:87] Created ranger Postgres database
I20260807 00:40:57.435041 1490 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-admin
2026-08-07 00:40:57,790 [I] DB FLAVOR :POSTGRES
2026-08-07 00:40:57,791 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:40:57,791 [I] Checking connection..
2026-08-07 00:40:57,791 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 00:40:58,074 [I] Checking connection passed.
2026-08-07 00:40:58,075 [I] --------- Verifying version history table ---------
2026-08-07 00:40:58,075 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:40:58,365 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:40:58,366 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:40:58,653 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:40:58,654 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 00:40:58,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 00:40:58.932 UTC [17208] WARNING: there is no transaction in progress
2026-08-07 00:40:58,944 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 00:40:58,945 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:40:59,213 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 00:40:59,213 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:40:59,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 00:40:59,498 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 00:40:59,769 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-07 00:41:00,038 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:41:00,329 [I] Table x_portal_user does not exist in database ranger
2026-08-07 00:41:00,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:41:00,611 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 00:41:00,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:41:00,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:41:01,174 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 00:41:01,175 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 00:41:02.126 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.145 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.162 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.179 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.194 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.214 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.266 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.273 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.287 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.300 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.310 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.323 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.332 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.339 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.345 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:02.352 UTC [17371] WARNING: there is no transaction in progress
2026-08-07 00:41:03,000 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 00:41:03,001 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 00:41:03,243 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 00:41:03,243 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:41:03,522 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 00:41:03,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:41:03,804 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 00:41:03,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:41:04,077 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:41:04,353 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-07 00:41:04,637 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 00:41:04,637 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 00:41:04,907 [I] DB_PATCHES have already been applied
I20260807 00:41:04.913123 1490 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLvHwjp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 00:41:04.913216 1490 mini_ranger.cc:193] Using host: 127.1.116.148
I20260807 00:41:04.917783 1490 mini_ranger.cc:227] Ranger admin URL: http://127.1.116.148:35319
Aug 07, 2026 12:41:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:41:05 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:41:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 12:41:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 12:41:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 12:41:05 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 12:41:05 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 00:41:06.282899 1490 mini_ranger.cc:160] Time spent starting Ranger: real 9.332s user 0.000s sys 0.015s
Aug 07, 2026 12:41:27 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 12:41:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 12:41:27 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 12:41:27 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 12:41:28 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 00:41:28.747848 1490 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/rangerkms_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/rangerkms_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.116.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.116.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 00:41:28 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063288, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 00:41:28.845795 1490 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 00:41:28.856397 1490 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 00:41:29.630511 1490 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 00:41:29.630666 1490 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms
I20260807 00:41:29.630681 1490 mini_ranger_kms.cc:227] Using postgres at 127.1.116.148:57265
2026-08-07 00:41:30,350 [I] DB FLAVOR :POSTGRES
2026-08-07 00:41:30,351 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:41:30,351 [I] Checking connection
2026-08-07 00:41:30,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 00:41:30,654 [I] connection success
2026-08-07 00:41:30,654 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 00:41:30,654 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 00:41:30,654 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 00:41:30,943 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 00:41:30,944 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:41:30,944 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 00:41:30,944 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:57265/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 00:41:31,246 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 00:41:31.774305 1490 mini_ranger_kms.cc:310] Created kms service
I20260807 00:41:31.998160 1490 mini_ranger_kms.cc:326] Created kudu user
I20260807 00:41:32.053037 1490 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 00:41:32 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063288, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:41:32.505910 1490 mini_ranger_kms.cc:384] Added ranger policy
I20260807 00:41:32.506035 1490 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 00:41:32.506062 1490 mini_ranger_kms.cc:240] Using host: 127.1.116.148
I20260807 00:41:32.510620 1490 mini_ranger_kms.cc:276] Ranger KMS PID: 17621
I20260807 00:41:32.510733 1490 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.116.148:54127
Aug 07, 2026 12:41:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:41:33 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:41:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 12:41:33 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 12:41:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 12:41:34 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 00:41:34 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786063294, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.116.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:41:35.231359 1490 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.386s user 0.010s sys 0.363s
I20260807 00:41:35.231743 1490 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 00:41:35.231856 1490 mini_ranger_kms.cc:401] 127.1.116.148:54127/kms/v1/keys
Aug 07 00:41:35 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063294, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.116.148@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
Aug 07 00:41:35 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063294, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.116.148@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
Aug 07 00:41:38 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:41:38.226226 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39117
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:33035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39117
--ranger_config_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:54127/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:41:38.339912 17684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:41:38.340134 17684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:41:38.340157 17684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:41:38.341317 17684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:41:38.341363 17684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:41:38.341378 17684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:41:38.341392 17684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:41:38.341404 17684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:41:38.343086 17684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:33035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.116.148:54127/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39117
--ranger_config_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:41:38.343343 17684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:41:38.343614 17684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:41:38.349418 17690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:38.349442 17691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:38.349484 17694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:41:38.349845 17684 server_base.cc:1034] running on GCE node
I20260807 00:41:38.350123 17684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:41:38.350445 17684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:41:38.351636 17684 hybrid_clock.cc:648] HybridClock initialized: now 1786063298351601 us; error 56 us; skew 500 ppm
Aug 07 00:41:38 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:41:38.359226 17684 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:41:38.359722 17684 webserver.cc:460] Webserver started at http://127.1.116.190:36675/ using document root <none> and password file <none>
I20260807 00:41:38.359982 17684 fs_manager.cc:359] Metadata directory not provided
I20260807 00:41:38.360042 17684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:41:38.360191 17684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:41:38 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063298, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:41:38.423467 17684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "085586fea3b544ab86cdfb17321ecc81"
format_stamp: "Formatted at 2026-08-07 00:41:38 on dist-test-slave-h6n0"
server_key: "95be1384dd010bf9ca7fe060b44de2ea"
server_key_iv: "39ab2fd7858259a31740c3598e9aedc6"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:38.424232 17684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "085586fea3b544ab86cdfb17321ecc81"
format_stamp: "Formatted at 2026-08-07 00:41:38 on dist-test-slave-h6n0"
server_key: "95be1384dd010bf9ca7fe060b44de2ea"
server_key_iv: "39ab2fd7858259a31740c3598e9aedc6"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:38.426476 17684 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260807 00:41:38.443997 17703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:38.444221 17684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:41:38.444344 17684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "085586fea3b544ab86cdfb17321ecc81"
format_stamp: "Formatted at 2026-08-07 00:41:38 on dist-test-slave-h6n0"
server_key: "95be1384dd010bf9ca7fe060b44de2ea"
server_key_iv: "39ab2fd7858259a31740c3598e9aedc6"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:38.444474 17684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:41:38.466966 17684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:41:38.469790 17684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:41:38.470010 17684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:41:38.479543 17684 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:39117
I20260807 00:41:38.479694 17768 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:39117 every 8 connection(s)
I20260807 00:41:38.480063 17684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:41:38.480964 17772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:41:38.484264 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81: Bootstrap starting.
I20260807 00:41:38.484649 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 17684
I20260807 00:41:38.484774 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:41:38.484982 1490 external_mini_cluster.cc:1428] Setting key bf9439aef72b21d3e055ca4a9e67c8c0
I20260807 00:41:38.485095 17772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81: Neither blocks nor log segments found. Creating new log.
I20260807 00:41:38.485800 17772 log.cc:824] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81: Log is configured to *not* fsync() on all Append() calls
I20260807 00:41:38.486953 17772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81: No bootstrap required, opened a new log
I20260807 00:41:38.490483 17772 raft_consensus.cc:348] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085586fea3b544ab86cdfb17321ecc81" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39117 } }
I20260807 00:41:38.490697 17772 raft_consensus.cc:374] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:41:38.490733 17772 raft_consensus.cc:729] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 085586fea3b544ab86cdfb17321ecc81, State: Initialized, Role: FOLLOWER
I20260807 00:41:38.490878 17772 consensus_queue.cc:260] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085586fea3b544ab86cdfb17321ecc81" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39117 } }
I20260807 00:41:38.490979 17772 raft_consensus.cc:388] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:41:38.491032 17772 raft_consensus.cc:482] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:41:38.491088 17772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:41:38.491742 17772 raft_consensus.cc:504] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085586fea3b544ab86cdfb17321ecc81" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39117 } }
I20260807 00:41:38.491869 17772 leader_election.cc:302] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 085586fea3b544ab86cdfb17321ecc81; no voters:
I20260807 00:41:38.492048 17772 leader_election.cc:288] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:41:38.492172 17777 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:41:38.492383 17772 sys_catalog.cc:564] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:41:38.492447 17777 raft_consensus.cc:686] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [term 1 LEADER]: Becoming Leader. State: Replica: 085586fea3b544ab86cdfb17321ecc81, State: Running, Role: LEADER
I20260807 00:41:38.492918 17777 consensus_queue.cc:237] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085586fea3b544ab86cdfb17321ecc81" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39117 } }
I20260807 00:41:38.493438 17777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 085586fea3b544ab86cdfb17321ecc81. Latest consensus state: current_term: 1 leader_uuid: "085586fea3b544ab86cdfb17321ecc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085586fea3b544ab86cdfb17321ecc81" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39117 } } }
I20260807 00:41:38.493633 17777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [sys.catalog]: This master's current role is: LEADER
I20260807 00:41:38.494400 17778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "085586fea3b544ab86cdfb17321ecc81" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "085586fea3b544ab86cdfb17321ecc81" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39117 } } }
I20260807 00:41:38.494500 17778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81 [sys.catalog]: This master's current role is: LEADER
I20260807 00:41:38.495126 17786 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:41:38.495587 17772 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 00:41:38.495972 17786 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 00:41:38 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:41:38.506055 17786 catalog_manager.cc:1347] Generated new cluster ID: a4d46cbf45ae40799e3497ab30b6dc87
I20260807 00:41:38.506233 17786 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:41:38.509065 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:38.485812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37758 (local address 127.1.116.190:39117)
0807 00:41:38.486241 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:38.486253 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:38.486297 (+ 44us) server_negotiation.cc:407] Connection header received
0807 00:41:38.486920 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:38.486930 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:38.487001 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:38.487302 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:41:38.488101 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:38.490879 (+ 2778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:38.491755 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:38.492514 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:38.492665 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:38.501354 (+ 8689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:41:38.501437 (+ 83us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:41:38.501437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:41:38.501648 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:41:38.506258 (+ 4610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:38.506995 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:38.507000 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:38.507001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:38.507113 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:38.507331 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:38.507332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:38.507332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:38.507875 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:41:38.508236 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:38.508318 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 00:41:38.508554 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":124,"threads_started":1}
I20260807 00:41:38.523164 17786 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:41:38.523949 17786 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:41:38.543818 17786 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81: Generated new TSK 0
I20260807 00:41:38.544126 17786 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:41:39.401331 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:54127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39117
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:33035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:41:39.544312 17826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:41:39.544593 17826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:41:39.544638 17826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:41:39.546105 17826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:41:39.546182 17826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:41:39.546231 17826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:41:39.548409 17826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:33035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.116.148:54127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:41:39.548743 17826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:41:39.549049 17826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:41:39.556191 17838 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:39.556669 17835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:39.557101 17834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:41:39.557283 17826 server_base.cc:1034] running on GCE node
I20260807 00:41:39.557534 17826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:41:39.557838 17826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:41:39.559091 17826 hybrid_clock.cc:648] HybridClock initialized: now 1786063299559046 us; error 59 us; skew 500 ppm
Aug 07 00:41:39 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:41:39.563642 17826 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:41:39.564258 17826 webserver.cc:460] Webserver started at http://127.1.116.129:34311/ using document root <none> and password file <none>
I20260807 00:41:39.564554 17826 fs_manager.cc:359] Metadata directory not provided
I20260807 00:41:39.564606 17826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:41:39.564764 17826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:41:39 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:41:39.587734 17826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "2773bb6291a94a0192fe8565492b36aa"
format_stamp: "Formatted at 2026-08-07 00:41:39 on dist-test-slave-h6n0"
server_key: "cdb6c27fc8f118642172a903d88ac7b2"
server_key_iv: "aa2a69363a895eac2f242e6ba5378427"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:39.588413 17826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "2773bb6291a94a0192fe8565492b36aa"
format_stamp: "Formatted at 2026-08-07 00:41:39 on dist-test-slave-h6n0"
server_key: "cdb6c27fc8f118642172a903d88ac7b2"
server_key_iv: "aa2a69363a895eac2f242e6ba5378427"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:39.590292 17826 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 00:41:39.605448 17849 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:39.605655 17826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:41:39.605775 17826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "2773bb6291a94a0192fe8565492b36aa"
format_stamp: "Formatted at 2026-08-07 00:41:39 on dist-test-slave-h6n0"
server_key: "cdb6c27fc8f118642172a903d88ac7b2"
server_key_iv: "aa2a69363a895eac2f242e6ba5378427"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:39.605906 17826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:41:39.617595 17826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:41:39.620522 17826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:41:39.620759 17826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:41:39.621158 17826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:41:39.621834 17826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:41:39.621938 17826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:39.622036 17826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:41:39.622128 17826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:39.638589 17826 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:46695
I20260807 00:41:39.639308 17826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:41:39.639947 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 17826
I20260807 00:41:39.640079 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:41:39.640307 1490 external_mini_cluster.cc:1428] Setting key e79ce855e2db324e0b588329f2a0ed98
I20260807 00:41:39.647275 17990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:46695 every 8 connection(s)
Aug 07 00:41:39 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:41:39.664654 17996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:39.646765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39117 (local address 127.1.116.129:52263)
0807 00:41:39.649917 (+ 3152us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:39.649974 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:39.650946 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:39.651400 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:39.651405 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:39.651727 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:41:39.652055 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:39.652061 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.653167 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.653168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:39.654501 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:39.654503 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.655073 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.655074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:39.655232 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:39.655951 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:41:39.655963 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:41:39.657864 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:41:39.662752 (+ 4888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:41:39.662761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:41:39.662764 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:41:39.663096 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:41:39.663343 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:41:39.663344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:41:39.663344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:41:39.663478 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:41:39.663784 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:41:39.663791 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:41:39.663978 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 00:41:39.664093 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 00:41:39.664365 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3095,"thread_start_us":160,"threads_started":1}
I20260807 00:41:39.665580 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:39.649069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:52263 (local address 127.1.116.190:39117)
0807 00:41:39.649825 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:39.649828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:39.650192 (+ 364us) server_negotiation.cc:407] Connection header received
0807 00:41:39.651143 (+ 951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:39.651144 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:39.651174 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:39.651246 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:41:39.652187 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.653050 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.654619 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.654935 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.654997 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:39.657987 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:41:39.657988 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:41:39.657988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:41:39.657996 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:41:39.662651 (+ 4655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:39.663198 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:39.663199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:39.663200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:39.663273 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:39.663546 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:39.663547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:39.663547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:39.663693 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:41:39.663757 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:39.665370 (+ 1613us) server_negotiation.cc:299] Negotiation successful
0807 00:41:39.665473 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":720,"thread_start_us":79,"threads_started":1}
I20260807 00:41:39.666478 17991 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39117
I20260807 00:41:39.666646 17991 heartbeater.cc:464] Registering TS with master...
I20260807 00:41:39.666953 17991 heartbeater.cc:511] Master 127.1.116.190:39117 requested a full tablet report, sending...
I20260807 00:41:39.667840 17730 ts_manager.cc:194] Registered new tserver with Master: 2773bb6291a94a0192fe8565492b36aa (127.1.116.129:46695)
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:41:39.668824 17730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:52263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:41:39.695466 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:54127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39117
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:33035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:41:39 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 00:41:39.851460 18001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:41:39.851694 18001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:41:39.851735 18001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:41:39.852880 18001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:41:39.852948 18001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:41:39.853001 18001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:41:39.855135 18001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:33035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.116.148:54127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:41:39.855397 18001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:41:39.855679 18001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:41:39.863420 18010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:39.863521 18014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:39.863582 18008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:41:39.864941 18001 server_base.cc:1034] running on GCE node
I20260807 00:41:39.865245 18001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:41:39.865710 18001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:41:39.866926 18001 hybrid_clock.cc:648] HybridClock initialized: now 1786063299866902 us; error 41 us; skew 500 ppm
Aug 07 00:41:39 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:41:39.871331 18001 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:41:39.872031 18001 webserver.cc:460] Webserver started at http://127.1.116.130:45833/ using document root <none> and password file <none>
I20260807 00:41:39.872380 18001 fs_manager.cc:359] Metadata directory not provided
I20260807 00:41:39.872467 18001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:41:39.872691 18001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:41:39 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:41:39.887915 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "d2d9aab6207840e796219930aca9df35"
format_stamp: "Formatted at 2026-08-07 00:41:39 on dist-test-slave-h6n0"
server_key: "aa60f355c3c09dd00de882f6bb498305"
server_key_iv: "eb39558ebbce798282a2d4089eabd90c"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:39.888505 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "d2d9aab6207840e796219930aca9df35"
format_stamp: "Formatted at 2026-08-07 00:41:39 on dist-test-slave-h6n0"
server_key: "aa60f355c3c09dd00de882f6bb498305"
server_key_iv: "eb39558ebbce798282a2d4089eabd90c"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:39.890683 18001 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260807 00:41:39.905025 18023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:39.906738 18001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:41:39.906868 18001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "d2d9aab6207840e796219930aca9df35"
format_stamp: "Formatted at 2026-08-07 00:41:39 on dist-test-slave-h6n0"
server_key: "aa60f355c3c09dd00de882f6bb498305"
server_key_iv: "eb39558ebbce798282a2d4089eabd90c"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:39.906976 18001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:41:39.933218 18001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:41:39.935921 18001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:41:39.936085 18001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:41:39.936342 18001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:41:39.936944 18001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:41:39.937083 18001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:39.937167 18001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:41:39.937228 18001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:39.949599 18001 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:41515
I20260807 00:41:39.950258 18001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:41:39.950865 18181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:41515 every 8 connection(s)
I20260807 00:41:39.952371 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 18001
I20260807 00:41:39.952507 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:41:39.952762 1490 external_mini_cluster.cc:1428] Setting key 804ad97fe9eab7fa27c2a8dc9163a92f
Aug 07 00:41:39 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
I20260807 00:41:39.972831 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:39.952595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:47467 (local address 127.1.116.190:39117)
0807 00:41:39.952679 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:39.952683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:39.958164 (+ 5481us) server_negotiation.cc:407] Connection header received
0807 00:41:39.959552 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:39.959554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:39.959578 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:39.959617 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:41:39.962685 (+ 3068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.963380 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.964585 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.965006 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.965100 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:39.968826 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:41:39.968828 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:41:39.968829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:41:39.968840 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:41:39.970878 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:39.971342 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:39.971344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:39.971344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:39.971597 (+ 253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:39.971903 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:39.971904 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:39.971907 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:39.972050 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:41:39.972134 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:39.972468 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 00:41:39.972725 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 00:41:39.972990 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:39.952278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39117 (local address 127.1.116.130:47467)
0807 00:41:39.957957 (+ 5679us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:39.958023 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:39.959360 (+ 1337us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:39.959858 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:39.959863 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:39.960327 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:41:39.960824 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:39.960830 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.963491 (+ 2661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.963491 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:39.964472 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:39.964474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:39.965253 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:39.965254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:39.965454 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:39.966231 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:41:39.966258 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:41:39.968691 (+ 2433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:41:39.970979 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:41:39.970988 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:41:39.970991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:41:39.971248 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:41:39.971681 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:41:39.971683 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:41:39.971683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:41:39.971821 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:41:39.972126 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:41:39.972134 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:41:39.972309 (+ 175us) client_negotiation.cc:769] Sending connection context
0807 00:41:39.972433 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 00:41:39.972627 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5618,"thread_start_us":98,"threads_started":1}
I20260807 00:41:39.973829 18182 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39117
I20260807 00:41:39.973948 18182 heartbeater.cc:464] Registering TS with master...
I20260807 00:41:39.974192 18182 heartbeater.cc:511] Master 127.1.116.190:39117 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
I20260807 00:41:39.974789 17730 ts_manager.cc:194] Registered new tserver with Master: d2d9aab6207840e796219930aca9df35 (127.1.116.130:41515)
I20260807 00:41:39.975337 17730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:47467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:41:39.991109 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:54127/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39117
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:33035
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:41:40.122486 18187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:41:40.122738 18187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:41:40.122774 18187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:41:40.123968 18187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:41:40.124014 18187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:41:40.124040 18187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:41:40.125639 18187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:33035
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.116.148:54127/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:41:40.125860 18187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:41:40.126106 18187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:41:40.132411 18193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:40.132485 18194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:41:40.132450 18196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:41:40.133653 18187 server_base.cc:1034] running on GCE node
I20260807 00:41:40.134150 18187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:41:40.134747 18187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:41:40.135998 18187 hybrid_clock.cc:648] HybridClock initialized: now 1786063300135964 us; error 39 us; skew 500 ppm
Aug 07 00:41:40 dist-test-slave-h6n0 krb5kdc[17067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063300, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:41:40.141345 18187 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:41:40.141793 18187 webserver.cc:460] Webserver started at http://127.1.116.131:45765/ using document root <none> and password file <none>
I20260807 00:41:40.142035 18187 fs_manager.cc:359] Metadata directory not provided
I20260807 00:41:40.142107 18187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:41:40.142256 18187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:41:40 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063300, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:41:40.169937 18187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "696aa647c5564056a615ea7468dcae89"
format_stamp: "Formatted at 2026-08-07 00:41:40 on dist-test-slave-h6n0"
server_key: "46a144b1ed9e31ea039314b9aaecb6c9"
server_key_iv: "aa94907347e6f28f6b8b673a2c9d3323"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:40.170553 18187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "696aa647c5564056a615ea7468dcae89"
format_stamp: "Formatted at 2026-08-07 00:41:40 on dist-test-slave-h6n0"
server_key: "46a144b1ed9e31ea039314b9aaecb6c9"
server_key_iv: "aa94907347e6f28f6b8b673a2c9d3323"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:40.177709 18187 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260807 00:41:40.196503 18204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:40.196674 18187 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.001s
I20260807 00:41:40.196768 18187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "696aa647c5564056a615ea7468dcae89"
format_stamp: "Formatted at 2026-08-07 00:41:40 on dist-test-slave-h6n0"
server_key: "46a144b1ed9e31ea039314b9aaecb6c9"
server_key_iv: "aa94907347e6f28f6b8b673a2c9d3323"
server_key_version: "kuduclusterkey@0"
I20260807 00:41:40.196856 18187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:41:40.209690 18187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:41:40.213967 18187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:41:40.214169 18187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:41:40.214469 18187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:41:40.214949 18187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:41:40.215004 18187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:40.215034 18187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:41:40.215052 18187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:41:40.230233 18187 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44869
I20260807 00:41:40.230919 18187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:41:40.239319 18365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44869 every 8 connection(s)
I20260807 00:41:40.240876 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 18187
I20260807 00:41:40.240996 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:41:40.241204 1490 external_mini_cluster.cc:1428] Setting key 6c8b6e9bc7b41bc029b93e9380c69ce3
Aug 07 00:41:40 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063300, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:41:40.251346 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.240209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:47741 (local address 127.1.116.190:39117)
0807 00:41:40.240308 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.240311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.240750 (+ 439us) server_negotiation.cc:407] Connection header received
0807 00:41:40.241628 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.241629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.241644 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.241677 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:41:40.242608 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.243386 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.244735 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.244951 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.245029 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.247775 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:41:40.247776 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:41:40.247777 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:41:40.247785 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:41:40.249674 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:40.250175 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:40.250176 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:40.250176 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:40.250239 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:40.250587 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:40.250589 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:40.250589 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:40.250725 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:41:40.250774 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.251159 (+ 385us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.251254 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:41:40.252079 18368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.240213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39117 (local address 127.1.116.131:47741)
0807 00:41:40.240548 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:40.240593 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:40.241473 (+ 880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:40.241882 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:40.241887 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:40.242163 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:41:40.242475 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.242481 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.243500 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.243501 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.244622 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.244625 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.245036 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.245037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.245208 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.245800 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:41:40.245809 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:41:40.247603 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:41:40.249789 (+ 2186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:41:40.249795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:41:40.249797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:41:40.250082 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:41:40.250321 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:41:40.250322 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:41:40.250322 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:41:40.250457 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:41:40.250826 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:41:40.250833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:41:40.250998 (+ 165us) client_negotiation.cc:769] Sending connection context
0807 00:41:40.251675 (+ 677us) client_negotiation.cc:240] Negotiation successful
0807 00:41:40.251851 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":197,"threads_started":1}
I20260807 00:41:40.253213 18366 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39117
I20260807 00:41:40.253304 18366 heartbeater.cc:464] Registering TS with master...
I20260807 00:41:40.253485 18366 heartbeater.cc:511] Master 127.1.116.190:39117 requested a full tablet report, sending...
I20260807 00:41:40.253875 17730 ts_manager.cc:194] Registered new tserver with Master: 696aa647c5564056a615ea7468dcae89 (127.1.116.131:44869)
I20260807 00:41:40.254242 17730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:47741
I20260807 00:41:40.263551 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:41:40.272042 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.265176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.1.116.190:39117)
0807 00:41:40.265216 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.265218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.265429 (+ 211us) server_negotiation.cc:407] Connection header received
0807 00:41:40.265505 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.265505 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.265516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.265571 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:41:40.266197 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.266734 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.267442 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.267702 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.267775 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.269130 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:41:40.269131 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:41:40.269132 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:41:40.269139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:41:40.270622 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:40.271021 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:40.271022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:40.271022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:40.271101 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:41:40.271335 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:41:40.271335 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:41:40.271336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:41:40.271468 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:41:40.271544 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.271791 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.271917 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260807 00:41:40.282727 17730 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:41:40.285061 17730 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:41:40.302472 18387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.288355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44869 (local address 127.0.0.1:55788)
0807 00:41:40.289417 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:40.289464 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:40.289561 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:40.289970 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:40.289974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:40.289977 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:41:40.290066 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.290068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.291482 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.291483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.293086 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.293090 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.302181 (+ 9091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.302182 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.302243 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.302248 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:41:40.302299 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 00:41:40.302373 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1024,"thread_start_us":107,"threads_started":1}
I20260807 00:41:40.302882 18388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.288853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55788 (local address 127.1.116.131:44869)
0807 00:41:40.289330 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.289335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.289659 (+ 324us) server_negotiation.cc:407] Connection header received
0807 00:41:40.289718 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.289721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.289740 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.289827 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:41:40.290186 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.291377 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.293241 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.294150 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.294243 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.294332 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.302734 (+ 8402us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.302791 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":112,"threads_started":1}
I20260807 00:41:40.303366 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.288856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.1.116.130:41515)
0807 00:41:40.289205 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.289212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.294488 (+ 5276us) server_negotiation.cc:407] Connection header received
0807 00:41:40.294696 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.294700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.294738 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.294804 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:41:40.295063 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.296195 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.298906 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.299737 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.299791 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.299869 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.303233 (+ 3364us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.303281 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":128,"threads_started":1}
I20260807 00:41:40.303643 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.288587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:41515 (local address 127.0.0.1:43000)
0807 00:41:40.294403 (+ 5816us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:40.294415 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:40.294555 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:40.294925 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:40.294926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:40.294929 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:41:40.294981 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.294982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.297498 (+ 2516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.297499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.298645 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.298649 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.301669 (+ 3020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.301670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.301761 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.301777 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 00:41:40.303515 (+ 1738us) client_negotiation.cc:240] Negotiation successful
0807 00:41:40.303569 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5792,"thread_start_us":719,"threads_started":1}
I20260807 00:41:40.303717 18260 tablet_service.cc:1467] Processing CreateTablet for tablet dd061c968e0f4cbaa3b96512c06bc36a (DEFAULT_TABLE table=test-table [id=7df283175df84c61896dfa63c1cf1916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:41:40.304116 18260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd061c968e0f4cbaa3b96512c06bc36a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:41:40.304626 18091 tablet_service.cc:1467] Processing CreateTablet for tablet dd061c968e0f4cbaa3b96512c06bc36a (DEFAULT_TABLE table=test-table [id=7df283175df84c61896dfa63c1cf1916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:41:40.305068 18091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd061c968e0f4cbaa3b96512c06bc36a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:41:40.311197 18395 tablet_bootstrap.cc:492] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Bootstrap starting.
I20260807 00:41:40.312618 18395 tablet_bootstrap.cc:654] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Neither blocks nor log segments found. Creating new log.
I20260807 00:41:40.313083 18394 tablet_bootstrap.cc:492] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Bootstrap starting.
I20260807 00:41:40.313853 18395 log.cc:824] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Log is configured to *not* fsync() on all Append() calls
I20260807 00:41:40.314664 18394 tablet_bootstrap.cc:654] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Neither blocks nor log segments found. Creating new log.
I20260807 00:41:40.315022 18394 log.cc:824] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Log is configured to *not* fsync() on all Append() calls
I20260807 00:41:40.315680 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.289045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.1.116.129:46695)
0807 00:41:40.298814 (+ 9769us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.298820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.310753 (+ 11933us) server_negotiation.cc:407] Connection header received
0807 00:41:40.310811 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.310814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.310836 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.311158 (+ 322us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:41:40.311242 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.312419 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.314315 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.315223 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.315332 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.315398 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.315503 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.315555 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9715,"thread_start_us":72,"threads_started":1}
I20260807 00:41:40.315941 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.288606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:46695 (local address 127.0.0.1:40686)
0807 00:41:40.310631 (+ 22025us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:40.310647 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:40.310724 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:40.311004 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:40.311005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:40.311008 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:41:40.311063 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.311064 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.312570 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.312571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.314193 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.314197 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.315337 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.315338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.315396 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.315401 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:41:40.315808 (+ 407us) client_negotiation.cc:240] Negotiation successful
0807 00:41:40.315871 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22009,"thread_start_us":619,"threads_started":1}
I20260807 00:41:40.316416 18394 tablet_bootstrap.cc:492] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: No bootstrap required, opened a new log
I20260807 00:41:40.316502 18394 ts_tablet_manager.cc:1397] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 00:41:40.316828 17896 tablet_service.cc:1467] Processing CreateTablet for tablet dd061c968e0f4cbaa3b96512c06bc36a (DEFAULT_TABLE table=test-table [id=7df283175df84c61896dfa63c1cf1916]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:41:40.317277 17896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd061c968e0f4cbaa3b96512c06bc36a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:41:40.318773 18395 tablet_bootstrap.cc:492] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: No bootstrap required, opened a new log
I20260807 00:41:40.318852 18395 ts_tablet_manager.cc:1397] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260807 00:41:40.322484 18399 tablet_bootstrap.cc:492] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa: Bootstrap starting.
I20260807 00:41:40.323945 18399 tablet_bootstrap.cc:654] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa: Neither blocks nor log segments found. Creating new log.
I20260807 00:41:40.324486 18399 log.cc:824] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa: Log is configured to *not* fsync() on all Append() calls
I20260807 00:41:40.325170 18395 raft_consensus.cc:348] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.325436 18395 raft_consensus.cc:374] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:41:40.325475 18395 raft_consensus.cc:729] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2d9aab6207840e796219930aca9df35, State: Initialized, Role: FOLLOWER
I20260807 00:41:40.325608 18395 consensus_queue.cc:260] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.325914 18395 ts_tablet_manager.cc:1428] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Time spent starting tablet: real 0.007s user 0.000s sys 0.005s
I20260807 00:41:40.326108 18182 heartbeater.cc:503] Master 127.1.116.190:39117 was elected leader, sending a full tablet report...
I20260807 00:41:40.332518 18394 raft_consensus.cc:348] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.332813 18394 raft_consensus.cc:374] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:41:40.332919 18394 raft_consensus.cc:729] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696aa647c5564056a615ea7468dcae89, State: Initialized, Role: FOLLOWER
I20260807 00:41:40.333151 18394 consensus_queue.cc:260] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.333570 18394 ts_tablet_manager.cc:1428] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Time spent starting tablet: real 0.017s user 0.000s sys 0.004s
I20260807 00:41:40.333801 18366 heartbeater.cc:503] Master 127.1.116.190:39117 was elected leader, sending a full tablet report...
I20260807 00:41:40.334523 18399 tablet_bootstrap.cc:492] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa: No bootstrap required, opened a new log
I20260807 00:41:40.334630 18399 ts_tablet_manager.cc:1397] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.002s
I20260807 00:41:40.338130 18399 raft_consensus.cc:348] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.338296 18399 raft_consensus.cc:374] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:41:40.338330 18399 raft_consensus.cc:729] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2773bb6291a94a0192fe8565492b36aa, State: Initialized, Role: FOLLOWER
I20260807 00:41:40.338613 18399 consensus_queue.cc:260] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.338989 18399 ts_tablet_manager.cc:1428] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa: Time spent starting tablet: real 0.004s user 0.003s sys 0.001s
I20260807 00:41:40.339241 17991 heartbeater.cc:503] Master 127.1.116.190:39117 was elected leader, sending a full tablet report...
Aug 07 00:41:40 dist-test-slave-h6n0 krb5kdc[17067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063299, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:41:40.362485 18402 raft_consensus.cc:482] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:41:40.362654 18402 raft_consensus.cc:504] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.363029 18402 leader_election.cc:288] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 696aa647c5564056a615ea7468dcae89 (127.1.116.131:44869), 2773bb6291a94a0192fe8565492b36aa (127.1.116.129:46695)
I20260807 00:41:40.370071 18391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.365707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:52539 (local address 127.1.116.129:46695)
0807 00:41:40.366074 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.366077 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.366089 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:41:40.366123 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.366124 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.366141 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.366201 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:41:40.366438 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.367269 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.368889 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.369599 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.369648 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.369658 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.369952 (+ 294us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.369998 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260807 00:41:40.370307 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.365513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44869 (local address 127.1.116.130:58375)
0807 00:41:40.365905 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:40.365918 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:40.365982 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:40.366405 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:40.366406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:40.366408 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:41:40.366639 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.366643 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.367706 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.367707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.369055 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.369058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.369807 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.369808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.370005 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.370010 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:41:40.370068 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 00:41:40.370131 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371}
I20260807 00:41:40.370674 18388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.365585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:58375 (local address 127.1.116.131:44869)
0807 00:41:40.365674 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.365678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.366253 (+ 575us) server_negotiation.cc:407] Connection header received
0807 00:41:40.366287 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.366288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.366299 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.366347 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:41:40.366735 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.367558 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.369162 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.369681 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.369761 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.369773 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.370518 (+ 745us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.370603 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:41:40.370939 18405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.365638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:46695 (local address 127.1.116.130:52539)
0807 00:41:40.365939 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:40.365953 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:41:40.366023 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:41:40.366228 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:41:40.366229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:41:40.366231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:41:40.366323 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.366325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.367417 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.367417 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.368779 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:41:40.368783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.369723 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.369724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:41:40.369858 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.369863 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:41:40.370371 (+ 508us) client_negotiation.cc:240] Negotiation successful
0807 00:41:40.370573 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":213,"threads_started":1}
I20260807 00:41:40.371166 18308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd061c968e0f4cbaa3b96512c06bc36a" candidate_uuid: "d2d9aab6207840e796219930aca9df35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696aa647c5564056a615ea7468dcae89" is_pre_election: true
I20260807 00:41:40.371383 18308 raft_consensus.cc:2454] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2d9aab6207840e796219930aca9df35 in term 0.
I20260807 00:41:40.371760 18026 leader_election.cc:302] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 696aa647c5564056a615ea7468dcae89, d2d9aab6207840e796219930aca9df35; no voters:
I20260807 00:41:40.371989 18402 raft_consensus.cc:2781] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:41:40.372061 18402 raft_consensus.cc:482] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:41:40.372085 18402 raft_consensus.cc:3037] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:41:40.373481 18402 raft_consensus.cc:504] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.373657 18402 leader_election.cc:288] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [CANDIDATE]: Term 1 election: Requested vote from peers 696aa647c5564056a615ea7468dcae89 (127.1.116.131:44869), 2773bb6291a94a0192fe8565492b36aa (127.1.116.129:46695)
I20260807 00:41:40.374003 17935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd061c968e0f4cbaa3b96512c06bc36a" candidate_uuid: "d2d9aab6207840e796219930aca9df35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2773bb6291a94a0192fe8565492b36aa"
I20260807 00:41:40.374009 17936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd061c968e0f4cbaa3b96512c06bc36a" candidate_uuid: "d2d9aab6207840e796219930aca9df35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2773bb6291a94a0192fe8565492b36aa" is_pre_election: true
I20260807 00:41:40.374213 17935 raft_consensus.cc:3037] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:41:40.375769 17935 raft_consensus.cc:2454] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d9aab6207840e796219930aca9df35 in term 1.
I20260807 00:41:40.376143 18308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd061c968e0f4cbaa3b96512c06bc36a" candidate_uuid: "d2d9aab6207840e796219930aca9df35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "696aa647c5564056a615ea7468dcae89"
I20260807 00:41:40.376252 18308 raft_consensus.cc:3037] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:41:40.377655 18308 raft_consensus.cc:2454] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2d9aab6207840e796219930aca9df35 in term 1.
I20260807 00:41:40.377962 18026 leader_election.cc:302] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 696aa647c5564056a615ea7468dcae89, d2d9aab6207840e796219930aca9df35; no voters:
I20260807 00:41:40.378276 18402 raft_consensus.cc:2781] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:41:40.378477 18402 raft_consensus.cc:686] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 1 LEADER]: Becoming Leader. State: Replica: d2d9aab6207840e796219930aca9df35, State: Running, Role: LEADER
I20260807 00:41:40.378620 18402 consensus_queue.cc:237] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } }
I20260807 00:41:40.379907 17730 catalog_manager.cc:5495] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2d9aab6207840e796219930aca9df35 (127.1.116.130). New cstate: current_term: 1 leader_uuid: "d2d9aab6207840e796219930aca9df35" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2d9aab6207840e796219930aca9df35" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41515 } health_report { overall_health: HEALTHY } } }
W20260807 00:41:40.396986 17992 tablet.cc:2369] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:41:40.399688 18389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.395506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43012 (local address 127.1.116.130:41515)
0807 00:41:40.395552 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0807 00:41:40.395554 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:41:40.395561 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:41:40.395597 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:41:40.395598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:41:40.395620 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:41:40.395683 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:41:40.395877 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.396621 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.398008 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:41:40.398314 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:41:40.398377 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:41:40.398857 (+ 480us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:41:40.399056 (+ 199us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:41:40.399152 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 00:41:40.399582 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 00:41:40.399605 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:41:40.402302 17935 raft_consensus.cc:1261] T dd061c968e0f4cbaa3b96512c06bc36a P 2773bb6291a94a0192fe8565492b36aa [term 1 FOLLOWER]: Refusing update from remote peer d2d9aab6207840e796219930aca9df35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:41:40.402935 18308 raft_consensus.cc:1261] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 1 FOLLOWER]: Refusing update from remote peer d2d9aab6207840e796219930aca9df35: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:41:40.402988 18406 consensus_queue.cc:1035] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2773bb6291a94a0192fe8565492b36aa" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 46695 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:41:40.405171 18406 consensus_queue.cc:1035] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [LEADER]: Connected to new peer: Peer: permanent_uuid: "696aa647c5564056a615ea7468dcae89" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44869 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:41:40.409489 18409 mvcc.cc:205] Tried to move back new op lower bound from 7315715278444220416 to 7315715278351450112. Current Snapshot: MvccSnapshot[applied={T|T < 7315715278444220416}]
I20260807 00:41:40.410223 18412 mvcc.cc:205] Tried to move back new op lower bound from 7315715278444220416 to 7315715278351450112. Current Snapshot: MvccSnapshot[applied={T|T < 7315715278444220416}]
I20260807 00:41:40.414203 18410 mvcc.cc:205] Tried to move back new op lower bound from 7315715278444220416 to 7315715278351450112. Current Snapshot: MvccSnapshot[applied={T|T < 7315715278444220416}]
I20260807 00:41:40.418047 17730 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:41:40.418198 17730 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:41:40.419303 17730 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 085586fea3b544ab86cdfb17321ecc81: Sending DeleteTablet for 3 replicas of tablet dd061c968e0f4cbaa3b96512c06bc36a
I20260807 00:41:40.419744 18091 tablet_service.cc:1514] Processing DeleteTablet for tablet dd061c968e0f4cbaa3b96512c06bc36a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:41:40 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:43000
I20260807 00:41:40.420591 18260 tablet_service.cc:1514] Processing DeleteTablet for tablet dd061c968e0f4cbaa3b96512c06bc36a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:41:40 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:55788
I20260807 00:41:40.420908 17896 tablet_service.cc:1514] Processing DeleteTablet for tablet dd061c968e0f4cbaa3b96512c06bc36a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:41:40 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:40686
I20260807 00:41:40.421686 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 17826
I20260807 00:41:40.426671 18421 tablet_replica.cc:330] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: stopping tablet replica
I20260807 00:41:40.426858 18421 raft_consensus.cc:2229] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:41:40.427115 18421 raft_consensus.cc:2258] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:41:40.427429 18420 tablet_replica.cc:330] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: stopping tablet replica
I20260807 00:41:40.427551 18420 raft_consensus.cc:2229] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:41:40.427698 18420 raft_consensus.cc:2258] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:41:40.430847 18421 ts_tablet_manager.cc:1905] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:41:40.432643 18420 ts_tablet_manager.cc:1905] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:41:40.433166 18421 ts_tablet_manager.cc:1918] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:41:40.433231 18421 log.cc:1196] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/dd061c968e0f4cbaa3b96512c06bc36a
I20260807 00:41:40.434043 18421 ts_tablet_manager.cc:1939] T dd061c968e0f4cbaa3b96512c06bc36a P 696aa647c5564056a615ea7468dcae89: Deleting consensus metadata
I20260807 00:41:40.434577 18420 ts_tablet_manager.cc:1918] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:41:40.434643 18420 log.cc:1196] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/dd061c968e0f4cbaa3b96512c06bc36a
I20260807 00:41:40.434877 18420 ts_tablet_manager.cc:1939] T dd061c968e0f4cbaa3b96512c06bc36a P d2d9aab6207840e796219930aca9df35: Deleting consensus metadata
I20260807 00:41:40.435335 17711 catalog_manager.cc:4841] TS d2d9aab6207840e796219930aca9df35 (127.1.116.130:41515): tablet dd061c968e0f4cbaa3b96512c06bc36a (table test-table [id=7df283175df84c61896dfa63c1cf1916]) successfully deleted
I20260807 00:41:40.439648 17708 catalog_manager.cc:4841] TS 696aa647c5564056a615ea7468dcae89 (127.1.116.131:44869): tablet dd061c968e0f4cbaa3b96512c06bc36a (table test-table [id=7df283175df84c61896dfa63c1cf1916]) successfully deleted
W20260807 00:41:40.449194 17708 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:46695 (error 108)
I20260807 00:41:40.450320 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 18001
I20260807 00:41:40.471136 18392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:41:40.470894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:46695 (local address 127.0.0.1:40700)
0807 00:41:40.470977 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 00:41:40.471066 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:46695: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260807 00:41:40.471344 17708 catalog_manager.cc:4584] TS 2773bb6291a94a0192fe8565492b36aa (127.1.116.129:46695): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dd061c968e0f4cbaa3b96512c06bc36a: Network error: Client connection negotiation failed: client connection to 127.1.116.129:46695: connect: Connection refused (error 111)
I20260807 00:41:40.474148 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 18187
I20260807 00:41:40.492511 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 17684
I20260807 00:41:40.509956 1490 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 00:41:40.865043 1490 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 00:41:40.865163 1490 mini_ranger.cc:66] Stopping Ranger...
I20260807 00:41:41.219712 1490 mini_ranger.cc:68] Stopped Ranger
2026-08-07 00:41:41.220 UTC [17108] LOG: received smart shutdown request
2026-08-07 00:41:41.223 UTC [17108] LOG: background worker "logical replication launcher" (PID 17118) exited with exit code 1
2026-08-07 00:41:41.224 UTC [17112] LOG: shutting down
2026-08-07 00:41:41.386 UTC [17108] LOG: database system is shut down
2026-08-07T00:41:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49763 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:41:41 dist-test-slave-h6n0 krb5kdc[18439](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:41:41 dist-test-slave-h6n0 krb5kdc[18439](info): set up 2 sockets
Aug 07 00:41:41 dist-test-slave-h6n0 krb5kdc[18439](info): commencing operation
krb5kdc: starting...
W20260807 00:41:42.771154 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:41:42 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:41:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:41:42Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/rangeradmin_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/rangeradmin_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/rangerlookup_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/rangerlookup_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
I20260807 00:41:42.954875 1490 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskLvHwjp/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/postgres -l logfile start
2026-08-07 00:41:45.380 UTC [18483] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 00:41:45.380 UTC [18483] LOG: listening on IPv4 address "127.1.116.148", port 60129
2026-08-07 00:41:45.383 UTC [18483] LOG: listening on Unix socket "/tmp/.s.PGSQL.60129"
2026-08-07 00:41:45.407 UTC [18486] LOG: database system was shut down at 2026-08-07 00:41:44 UTC
2026-08-07 00:41:45.411 UTC [18483] LOG: database system is ready to accept connections
I20260807 00:41:46.568594 1490 mini_postgres.cc:96] Postgres bound to 60129
2026-08-07 00:41:46.577 UTC [18496] FATAL: database "slave" does not exist
127.1.116.148:60129 - accepting connections
I20260807 00:41:46.577741 1490 mini_ranger.cc:161] Starting Ranger...
I20260807 00:41:46.594828 1490 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 00:41:47.035269 1490 mini_ranger.cc:87] Created ranger Postgres database
I20260807 00:41:47.035383 1490 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-admin
2026-08-07 00:41:47,323 [I] DB FLAVOR :POSTGRES
2026-08-07 00:41:47,323 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:41:47,324 [I] Checking connection..
2026-08-07 00:41:47,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 00:41:47,580 [I] Checking connection passed.
2026-08-07 00:41:47,581 [I] --------- Verifying version history table ---------
2026-08-07 00:41:47,581 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:41:47,868 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:41:47,869 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:41:48,136 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:41:48,137 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 00:41:48,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 00:41:48.428 UTC [18590] WARNING: there is no transaction in progress
2026-08-07 00:41:48,438 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 00:41:48,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:41:48,698 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 00:41:48,698 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:41:48,699 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 00:41:48,994 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 00:41:49,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-h6n0.c.gcp-upstream.internal','N') ;"
2026-08-07 00:41:49,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:41:49,808 [I] Table x_portal_user does not exist in database ranger
2026-08-07 00:41:49,809 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:41:50,074 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 00:41:50,074 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:41:50,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:41:50,634 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 00:41:50,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 00:41:51.650 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.666 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.682 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.697 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.713 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.731 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.784 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.791 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.805 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.815 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.825 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.837 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.846 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.852 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.858 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:51.864 UTC [18738] WARNING: there is no transaction in progress
2026-08-07 00:41:52,488 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 00:41:52,488 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 00:41:52,742 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 00:41:52,743 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:41:53,024 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 00:41:53,024 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:41:53,286 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 00:41:53,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:41:53,550 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:41:53,863 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-h6n0.c.gcp-upstream.internal';"
2026-08-07 00:41:54,146 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 00:41:54,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 00:41:54,418 [I] DB_PATCHES have already been applied
I20260807 00:41:54.424290 1490 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-admin:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskLvHwjp/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 00:41:54.424389 1490 mini_ranger.cc:193] Using host: 127.1.116.148
I20260807 00:41:54.429026 1490 mini_ranger.cc:227] Ranger admin URL: http://127.1.116.148:39627
Aug 07, 2026 12:41:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:41:54 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:41:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 12:41:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 12:41:54 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 12:41:55 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 12:41:55 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 00:41:55.821404 1490 mini_ranger.cc:160] Time spent starting Ranger: real 9.244s user 0.000s sys 0.013s
Aug 07, 2026 12:42:17 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 12:42:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 12:42:17 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 12:42:17 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 12:42:18 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 00:42:19.165364 1490 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.116.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.116.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/rangerkms_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.116.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/rangerkms_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.116.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.116.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.116.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/HTTP_127.1.116.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 00:42:19 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063339, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 00:42:19.270296 1490 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 00:42:19.282507 1490 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 00:42:20.122072 1490 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 00:42:20.122170 1490 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms
I20260807 00:42:20.122187 1490 mini_ranger_kms.cc:227] Using postgres at 127.1.116.148:60129
2026-08-07 00:42:20,307 [I] DB FLAVOR :POSTGRES
2026-08-07 00:42:20,308 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:42:20,308 [I] Checking connection
2026-08-07 00:42:20,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 00:42:20,574 [I] connection success
2026-08-07 00:42:20,574 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 00:42:20,574 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 00:42:20,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 00:42:20,834 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 00:42:20,835 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:42:20,835 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 00:42:20,835 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.116.148:60129/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 00:42:21,141 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 00:42:21.721634 1490 mini_ranger_kms.cc:310] Created kms service
I20260807 00:42:21.908644 1490 mini_ranger_kms.cc:326] Created kudu user
I20260807 00:42:21.958616 1490 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 00:42:21 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063339, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:22.430414 1490 mini_ranger_kms.cc:384] Added ranger policy
I20260807 00:42:22.430737 1490 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms:/tmp/dist-test-taskLvHwjp/build/release/bin/postgresql.jar:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskLvHwjp/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 00:42:22.430768 1490 mini_ranger_kms.cc:240] Using host: 127.1.116.148
I20260807 00:42:22.436866 1490 mini_ranger_kms.cc:276] Ranger KMS PID: 18983
I20260807 00:42:22.436962 1490 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.116.148:40595
Aug 07, 2026 12:42:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:42:23 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:42:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 12:42:23 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 12:42:24 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 12:42:24 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 00:42:24 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786063344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.116.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:25.179029 1490 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.909s user 0.013s sys 0.154s
I20260807 00:42:25.179152 1490 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 00:42:25.179209 1490 mini_ranger_kms.cc:401] 127.1.116.148:40595/kms/v1/keys
Aug 07 00:42:25 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.116.148@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
Aug 07 00:42:25 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063344, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.116.148@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
Aug 07 00:42:28 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:42:28.615054 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41039
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:41039
--ranger_config_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:28.700701 19050 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:28.700902 19050 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:28.700923 19050 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:28.701960 19050 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:28.702000 19050 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:28.702014 19050 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:28.702028 19050 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:28.702040 19050 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:28.703510 19050 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.116.148:40595/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:41039
--ranger_config_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19050
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:28.703694 19050 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:28.703853 19050 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:28.709386 19057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:28.709470 19055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:28.709537 19050 server_base.cc:1034] running on GCE node
W20260807 00:42:28.709477 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:28.710103 19050 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:28.710444 19050 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:28.711614 19050 hybrid_clock.cc:648] HybridClock initialized: now 1786063348711596 us; error 37 us; skew 500 ppm
Aug 07 00:42:28 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:28.718711 19050 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:28.719260 19050 webserver.cc:460] Webserver started at http://127.1.116.190:40601/ using document root <none> and password file <none>
I20260807 00:42:28.719521 19050 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:28.719585 19050 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:28.719766 19050 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:42:28 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:28.781543 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "01e7f4df5cc14ac69089c7f94ff325be"
format_stamp: "Formatted at 2026-08-07 00:42:28 on dist-test-slave-h6n0"
server_key: "5579d1c946e6551b630a69a6e8a831ad"
server_key_iv: "715365029341200a145afdd26ee081bc"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:28.782039 19050 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "01e7f4df5cc14ac69089c7f94ff325be"
format_stamp: "Formatted at 2026-08-07 00:42:28 on dist-test-slave-h6n0"
server_key: "5579d1c946e6551b630a69a6e8a831ad"
server_key_iv: "715365029341200a145afdd26ee081bc"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:28.784050 19050 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 00:42:28.799014 19068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:28.799269 19050 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:28.799403 19050 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "01e7f4df5cc14ac69089c7f94ff325be"
format_stamp: "Formatted at 2026-08-07 00:42:28 on dist-test-slave-h6n0"
server_key: "5579d1c946e6551b630a69a6e8a831ad"
server_key_iv: "715365029341200a145afdd26ee081bc"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:28.799527 19050 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:28.820751 19050 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:28.822872 19050 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:28.823158 19050 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:28.828428 19050 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:41039
I20260807 00:42:28.828509 19128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:41039 every 8 connection(s)
I20260807 00:42:28.828946 19050 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:28.829808 19130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:28.832275 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 19050
I20260807 00:42:28.832369 19130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Bootstrap starting.
I20260807 00:42:28.832409 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:42:28.832615 1490 external_mini_cluster.cc:1428] Setting key 7f53fbe36ccc7f314920438cc2821b87
I20260807 00:42:28.833130 19130 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:28.833585 19130 log.cc:824] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:28.834444 19130 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: No bootstrap required, opened a new log
I20260807 00:42:28.836282 19130 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:28.836454 19130 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:28.836489 19130 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e7f4df5cc14ac69089c7f94ff325be, State: Initialized, Role: FOLLOWER
I20260807 00:42:28.836621 19130 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:28.836694 19130 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:42:28.836720 19130 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:42:28.836755 19130 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:28.837400 19130 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:28.837497 19130 leader_election.cc:302] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e7f4df5cc14ac69089c7f94ff325be; no voters:
I20260807 00:42:28.837715 19130 leader_election.cc:288] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:42:28.837752 19138 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:28.837906 19138 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 LEADER]: Becoming Leader. State: Replica: 01e7f4df5cc14ac69089c7f94ff325be, State: Running, Role: LEADER
I20260807 00:42:28.837989 19130 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:28.838665 19138 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:28.839078 19139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e7f4df5cc14ac69089c7f94ff325be. Latest consensus state: current_term: 1 leader_uuid: "01e7f4df5cc14ac69089c7f94ff325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } } }
I20260807 00:42:28.839339 19140 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e7f4df5cc14ac69089c7f94ff325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } } }
I20260807 00:42:28.839705 19140 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:28.839955 19139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:28.839974 19148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:28.840539 19148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:28.840929 19130 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 07 00:42:28 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063348, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:28.842396 19156 mvcc.cc:205] Tried to move back new op lower bound from 7315715476854636544 to 7315715476843667456. Current Snapshot: MvccSnapshot[applied={T|T < 7315715476854636544}]
I20260807 00:42:28.847728 19148 catalog_manager.cc:1347] Generated new cluster ID: 46f4dbc2c47b4b3dbdbb0a9059614d83
I20260807 00:42:28.847795 19148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:42:28.850320 19134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:28.833308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43646 (local address 127.1.116.190:41039)
0807 00:42:28.833564 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:28.833574 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:28.833611 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:42:28.834142 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:28.834150 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:28.834213 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:28.834371 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:28.835154 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:28.836977 (+ 1823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:28.838269 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:28.838881 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:28.839024 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:28.842226 (+ 3202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:28.842233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:28.842233 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:28.842249 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:28.848143 (+ 5894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:28.848601 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:28.848604 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:28.848605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:28.848715 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:28.848888 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:28.848888 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:28.848889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:28.849242 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:28.849799 (+ 557us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:28.849875 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 00:42:28.850046 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":104,"threads_started":1}
I20260807 00:42:28.856681 19148 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:42:28.857288 19148 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:42:28.863401 19148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Generated new TSK 0
I20260807 00:42:28.863612 19148 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:42:29.664992 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41039
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:29.783138 19186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:29.783358 19186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:29.783396 19186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:29.784642 19186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:29.784704 19186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:29.784742 19186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:42:29.786473 19186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.116.148:40595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:29.786756 19186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:29.787009 19186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:29.792511 19196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:29.792511 19200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:29.792510 19197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:29.793279 19186 server_base.cc:1034] running on GCE node
I20260807 00:42:29.793519 19186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:29.793776 19186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:29.794939 19186 hybrid_clock.cc:648] HybridClock initialized: now 1786063349794918 us; error 51 us; skew 500 ppm
Aug 07 00:42:29 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063349, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:29.798058 19186 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:42:29.798851 19186 webserver.cc:460] Webserver started at http://127.1.116.129:33373/ using document root <none> and password file <none>
I20260807 00:42:29.799172 19186 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:29.799242 19186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:29.799393 19186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:42:29 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063349, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:29.813876 19186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "9f5f66be4a1747e4a1fcdddc05424f9d"
format_stamp: "Formatted at 2026-08-07 00:42:29 on dist-test-slave-h6n0"
server_key: "ea9d83de502c9d17108097955cdc0a9d"
server_key_iv: "d2cdf06597f44df23ff3e5f073223b57"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:29.814399 19186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "9f5f66be4a1747e4a1fcdddc05424f9d"
format_stamp: "Formatted at 2026-08-07 00:42:29 on dist-test-slave-h6n0"
server_key: "ea9d83de502c9d17108097955cdc0a9d"
server_key_iv: "d2cdf06597f44df23ff3e5f073223b57"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:29.817224 19186 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.001s
I20260807 00:42:29.832898 19212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:29.833165 19186 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:29.833331 19186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "9f5f66be4a1747e4a1fcdddc05424f9d"
format_stamp: "Formatted at 2026-08-07 00:42:29 on dist-test-slave-h6n0"
server_key: "ea9d83de502c9d17108097955cdc0a9d"
server_key_iv: "d2cdf06597f44df23ff3e5f073223b57"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:29.833494 19186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:29.864797 19186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:29.867286 19186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:29.867515 19186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:29.867826 19186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:29.868255 19186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:29.868351 19186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:29.868449 19186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:29.868539 19186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:29.875368 19186 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:33175
I20260807 00:42:29.875838 19186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:42:29.882700 19360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:33175 every 8 connection(s)
I20260807 00:42:29.883419 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 19186
I20260807 00:42:29.883551 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:42:29.883755 1490 external_mini_cluster.cc:1428] Setting key c0b7a9f47a06b73d3aaabdbf76f620b7
Aug 07 00:42:29 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063349, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:29.895254 19366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:29.883491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41039 (local address 127.1.116.129:35051)
0807 00:42:29.883770 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:29.883832 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:29.884690 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:29.885072 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:29.885076 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:29.885316 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:29.885604 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:29.885609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:29.886576 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:29.886578 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:29.887904 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:29.887907 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:29.888545 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:29.888547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:29.888712 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:29.889359 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:29.889372 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:29.891389 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:29.893526 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:29.893534 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:29.893536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:29.893841 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:29.894093 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:29.894094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:29.894094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:29.894221 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:29.894553 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:29.894560 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:29.894730 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 00:42:29.894838 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 00:42:29.895013 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":120,"threads_started":1}
I20260807 00:42:29.895875 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:29.883515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:35051 (local address 127.1.116.190:41039)
0807 00:42:29.883695 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:29.883698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:29.884097 (+ 399us) server_negotiation.cc:407] Connection header received
0807 00:42:29.884870 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:29.884871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:29.884884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:29.884963 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:29.885706 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:29.886432 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:29.888037 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:29.888404 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:29.888523 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:29.891521 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:29.891523 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:29.891523 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:29.891531 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:29.893414 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:29.893929 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:29.893931 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:29.893931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:29.893997 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:29.894289 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:29.894290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:29.894290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:29.894429 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:29.894502 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:29.895689 (+ 1187us) server_negotiation.cc:299] Negotiation successful
0807 00:42:29.895776 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":111,"threads_started":1}
I20260807 00:42:29.896643 19361 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41039
I20260807 00:42:29.896806 19361 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:29.897066 19361 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:29.897715 19088 ts_manager.cc:194] Registered new tserver with Master: 9f5f66be4a1747e4a1fcdddc05424f9d (127.1.116.129:33175)
I20260807 00:42:29.898711 19088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:35051
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:42:29.926415 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41039
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260807 00:42:30.062440 19373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:30.062819 19373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:30.062873 19373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:30.064823 19373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:30.064890 19373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:30.064934 19373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:42:30.067276 19373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.116.148:40595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:30.067579 19373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:30.067880 19373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:30.077461 19386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.077461 19382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.077965 19383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:30.079211 19373 server_base.cc:1034] running on GCE node
I20260807 00:42:30.079490 19373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:30.079824 19373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:30.081003 19373 hybrid_clock.cc:648] HybridClock initialized: now 1786063350080980 us; error 43 us; skew 500 ppm
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:30.083854 19373 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:42:30.084355 19373 webserver.cc:460] Webserver started at http://127.1.116.130:44937/ using document root <none> and password file <none>
I20260807 00:42:30.084633 19373 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:30.084700 19373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:30.084841 19373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:30.099164 19373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "c9f17aa8416a41299f2daf81c7bfe2a3"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "e578128462217c20325acd77c7433e96"
server_key_iv: "1afc257937257a542b853c92c35bfbc2"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.099669 19373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "c9f17aa8416a41299f2daf81c7bfe2a3"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "e578128462217c20325acd77c7433e96"
server_key_iv: "1afc257937257a542b853c92c35bfbc2"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.102900 19373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:42:30.119915 19394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.120150 19373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:30.120254 19373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "c9f17aa8416a41299f2daf81c7bfe2a3"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "e578128462217c20325acd77c7433e96"
server_key_iv: "1afc257937257a542b853c92c35bfbc2"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.120366 19373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:30.137490 19373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:30.140476 19373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:30.140676 19373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:30.140990 19373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:30.141423 19373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:30.141518 19373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.141597 19373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:30.141662 19373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.148609 19373 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:42305
I20260807 00:42:30.149255 19373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:42:30.152073 19560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:42305 every 8 connection(s)
I20260807 00:42:30.156562 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 19373
I20260807 00:42:30.156724 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:42:30.156963 1490 external_mini_cluster.cc:1428] Setting key cf5238ae480b560a1870e75ded6914bc
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:30.166218 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:30.152801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:41807 (local address 127.1.116.190:41039)
0807 00:42:30.152877 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:30.152880 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:30.154813 (+ 1933us) server_negotiation.cc:407] Connection header received
0807 00:42:30.155552 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:30.155554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:30.155572 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:30.155606 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:30.156442 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.157207 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.159091 (+ 1884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.159393 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.159440 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:30.162455 (+ 3015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:30.162457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:30.162457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:30.162465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:30.164416 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:30.164966 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:30.164967 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:30.164967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:30.165035 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:30.165362 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:30.165364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:30.165364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:30.165513 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:30.165567 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:30.165984 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 00:42:30.166081 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 00:42:30.167068 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:30.153041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41039 (local address 127.1.116.130:41807)
0807 00:42:30.154677 (+ 1636us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:30.154724 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:30.155418 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:30.155796 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:30.155800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:30.156051 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:30.156321 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:30.156327 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.157594 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.157595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:30.158969 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:30.158975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.159556 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.159557 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:30.159693 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:30.160289 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:30.160300 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:30.162320 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:30.164533 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:30.164540 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:30.164542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:30.164856 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:30.165120 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:30.165121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:30.165121 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:30.165261 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:30.165628 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:30.165636 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:30.165793 (+ 157us) client_negotiation.cc:769] Sending connection context
0807 00:42:30.166640 (+ 847us) client_negotiation.cc:240] Negotiation successful
0807 00:42:30.166811 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1589,"thread_start_us":153,"threads_started":1}
I20260807 00:42:30.167857 19561 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41039
I20260807 00:42:30.167963 19561 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:30.168190 19561 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:30.168668 19088 ts_manager.cc:194] Registered new tserver with Master: c9f17aa8416a41299f2daf81c7bfe2a3 (127.1.116.130:42305)
I20260807 00:42:30.169157 19088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:41807
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:42:30.217680 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41039
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:30.379585 19568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:30.379927 19568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:30.380038 19568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:30.381678 19568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:30.381778 19568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:30.381867 19568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:42:30.384686 19568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.116.148:40595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:30.385139 19568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:30.385463 19568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:30.392469 19574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.392843 19573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.393397 19577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:30.393908 19568 server_base.cc:1034] running on GCE node
I20260807 00:42:30.394138 19568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:30.394379 19568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:30.395546 19568 hybrid_clock.cc:648] HybridClock initialized: now 1786063350395516 us; error 43 us; skew 500 ppm
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:30.399396 19568 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:42:30.399961 19568 webserver.cc:460] Webserver started at http://127.1.116.131:35089/ using document root <none> and password file <none>
I20260807 00:42:30.400305 19568 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:30.400456 19568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:30.400647 19568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:30.425436 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "e3f1770567494386b807e1f887739892"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "1c4255736c2006f7a91cdce66bcb7b6b"
server_key_iv: "52be0579c34328f29e6b9542030f9948"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.426011 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "e3f1770567494386b807e1f887739892"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "1c4255736c2006f7a91cdce66bcb7b6b"
server_key_iv: "52be0579c34328f29e6b9542030f9948"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.434892 19568 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260807 00:42:30.452606 19586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.452895 19568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260807 00:42:30.453042 19568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "e3f1770567494386b807e1f887739892"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "1c4255736c2006f7a91cdce66bcb7b6b"
server_key_iv: "52be0579c34328f29e6b9542030f9948"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.453210 19568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:30.475072 19568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:30.477602 19568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:30.477799 19568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:30.478098 19568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:30.478566 19568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:30.478663 19568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.478750 19568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:30.478881 19568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.489584 19568 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:41951
I20260807 00:42:30.490269 19568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:42:30.493057 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 19568
I20260807 00:42:30.493198 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:42:30.493417 1490 external_mini_cluster.cc:1428] Setting key 36687f59460a2cdd8336f6cc41e15141
I20260807 00:42:30.497145 19755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:41951 every 8 connection(s)
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:30.533203 19367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:30.509718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:36643 (local address 127.1.116.190:41039)
0807 00:42:30.510457 (+ 739us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:30.510459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:30.518938 (+ 8479us) server_negotiation.cc:407] Connection header received
0807 00:42:30.519657 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:30.519659 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:30.519673 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:30.519739 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:30.520771 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.521554 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.523993 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.524303 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.524359 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:30.528885 (+ 4526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:30.528887 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:30.528887 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:30.528975 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:30.530773 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:30.531947 (+ 1174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:30.531948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:30.531949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:30.532015 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:30.532374 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:30.532375 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:30.532376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:30.532538 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:30.532594 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:30.533001 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 00:42:30.533089 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708}
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:30.535389 19761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:30.509295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41039 (local address 127.1.116.131:36643)
0807 00:42:30.518744 (+ 9449us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:30.518822 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:30.519518 (+ 696us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:30.519915 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:30.519922 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:30.520275 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:30.520639 (+ 364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:30.520647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.521690 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.521691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:30.523808 (+ 2117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:30.523819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:30.524490 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:30.524491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:30.524666 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:30.525485 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:30.525497 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:30.528768 (+ 3271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:30.531503 (+ 2735us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:30.531511 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:30.531514 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:30.531842 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:30.532089 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:30.532090 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:30.532091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:30.532288 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:30.532640 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:30.532647 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:30.532834 (+ 187us) client_negotiation.cc:769] Sending connection context
0807 00:42:30.533590 (+ 756us) client_negotiation.cc:240] Negotiation successful
0807 00:42:30.533780 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9359,"thread_start_us":128,"threads_started":1}
I20260807 00:42:30.536219 19756 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41039
I20260807 00:42:30.536419 19756 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:30.536702 19756 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:30.537359 19088 ts_manager.cc:194] Registered new tserver with Master: e3f1770567494386b807e1f887739892 (127.1.116.131:41951)
I20260807 00:42:30.537859 19088 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:36643
I20260807 00:42:30.539862 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:42:30.540876 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 19186
I20260807 00:42:30.560297 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 19373
I20260807 00:42:30.568673 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 19568
I20260807 00:42:30.578704 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 19050
I20260807 00:42:30.596585 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41039
--webserver_interface=127.1.116.190
--webserver_port=40601
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:41039
--ranger_config_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:30.717373 19762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:30.717623 19762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:30.717650 19762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:30.718812 19762 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:30.718856 19762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:30.718871 19762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:30.718884 19762 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:30.718896 19762 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:30.720472 19762 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:41039
--ranger_config_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=40601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:30.720695 19762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:30.720888 19762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:30.726385 19769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.726343 19771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.726349 19767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:30.726853 19762 server_base.cc:1034] running on GCE node
I20260807 00:42:30.727178 19762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:30.727519 19762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:30.728681 19762 hybrid_clock.cc:648] HybridClock initialized: now 1786063350728666 us; error 39 us; skew 500 ppm
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:30.731608 19762 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:30.732124 19762 webserver.cc:460] Webserver started at http://127.1.116.190:40601/ using document root <none> and password file <none>
I20260807 00:42:30.732376 19762 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:30.732465 19762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:30.779812 19762 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:30.780737 19781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.781009 19762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:30.781109 19762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "01e7f4df5cc14ac69089c7f94ff325be"
format_stamp: "Formatted at 2026-08-07 00:42:28 on dist-test-slave-h6n0"
server_key: "5579d1c946e6551b630a69a6e8a831ad"
server_key_iv: "715365029341200a145afdd26ee081bc"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.781503 19762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:30.794806 19762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:30.796569 19762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:30.796713 19762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:30.801575 19762 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:41039
I20260807 00:42:30.801625 19861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:41039 every 8 connection(s)
I20260807 00:42:30.802044 19762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:30.803639 19864 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:42:30.804512 19864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Bootstrap starting.
I20260807 00:42:30.810299 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 19762
I20260807 00:42:30.810657 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:33175
--local_ip_for_outbound_sockets=127.1.116.129
--tserver_master_addrs=127.1.116.190:41039
--webserver_port=33373
--webserver_interface=127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:42:30.814613 19864 log.cc:824] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:30.816592 19864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:42:30.816920 19864 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Bootstrap complete.
I20260807 00:42:30.819705 19864 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:30.820001 19864 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e7f4df5cc14ac69089c7f94ff325be, State: Initialized, Role: FOLLOWER
I20260807 00:42:30.820211 19864 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:30.820302 19864 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:42:30.820358 19864 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:42:30.820411 19864 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:42:30.821580 19864 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:30.821722 19864 leader_election.cc:302] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01e7f4df5cc14ac69089c7f94ff325be; no voters:
I20260807 00:42:30.821959 19864 leader_election.cc:288] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:42:30.821998 19868 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:42:30.822136 19868 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [term 2 LEADER]: Becoming Leader. State: Replica: 01e7f4df5cc14ac69089c7f94ff325be, State: Running, Role: LEADER
I20260807 00:42:30.822289 19864 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:30.822362 19868 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } }
I20260807 00:42:30.822824 19870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "01e7f4df5cc14ac69089c7f94ff325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } } }
I20260807 00:42:30.822925 19870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:30.823228 19871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e7f4df5cc14ac69089c7f94ff325be. Latest consensus state: current_term: 2 leader_uuid: "01e7f4df5cc14ac69089c7f94ff325be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01e7f4df5cc14ac69089c7f94ff325be" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41039 } } }
I20260807 00:42:30.823314 19871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:30.823558 19878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:30.824186 19878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:30.824326 19878 catalog_manager.cc:1259] Loaded cluster ID: 46f4dbc2c47b4b3dbdbb0a9059614d83
I20260807 00:42:30.824420 19878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:42:30.824723 19864 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 00:42:30.825734 19878 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:42:30.825944 19878 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Loaded TSK: 0
I20260807 00:42:30.826197 19878 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:42:30.924001 19866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:30.924269 19866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:30.924310 19866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:30.925431 19866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:30.925498 19866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:30.925549 19866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:42:30.927320 19866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:33175
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=33373
--tserver_master_addrs=127.1.116.190:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:30.927590 19866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:30.927811 19866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:30.933168 19921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.933507 19917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:30.933156 19918 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:30.934185 19866 server_base.cc:1034] running on GCE node
I20260807 00:42:30.934387 19866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:30.934657 19866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:30.942080 19866 hybrid_clock.cc:648] HybridClock initialized: now 1786063350942059 us; error 44 us; skew 500 ppm
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:30.947223 19866 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:42:30.947739 19866 webserver.cc:460] Webserver started at http://127.1.116.129:33373/ using document root <none> and password file <none>
I20260807 00:42:30.947986 19866 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:30.948053 19866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:42:30 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:30.967614 19866 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:42:30.969177 19931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.969377 19866 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:30.969485 19866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "9f5f66be4a1747e4a1fcdddc05424f9d"
format_stamp: "Formatted at 2026-08-07 00:42:29 on dist-test-slave-h6n0"
server_key: "ea9d83de502c9d17108097955cdc0a9d"
server_key_iv: "d2cdf06597f44df23ff3e5f073223b57"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:30.970072 19866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:30.983968 19866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:30.988443 19866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:30.988701 19866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:30.989001 19866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:30.989370 19866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:30.989416 19866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.989447 19866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:30.989465 19866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:30.997509 19866 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:33175
I20260807 00:42:30.999469 20088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:33175 every 8 connection(s)
I20260807 00:42:31.000350 19866 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:42:31.009243 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 19866
I20260807 00:42:31.009593 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:42305
--local_ip_for_outbound_sockets=127.1.116.130
--tserver_master_addrs=127.1.116.190:41039
--webserver_port=44937
--webserver_interface=127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:42:31 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:31.015610 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:31.001352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:43705 (local address 127.1.116.190:41039)
0807 00:42:31.001595 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:31.001606 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:31.001776 (+ 170us) server_negotiation.cc:407] Connection header received
0807 00:42:31.002667 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:31.002678 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:31.002768 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:31.003057 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:31.003922 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.005425 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.007208 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.007849 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.007988 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:31.010647 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:31.010652 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:31.010653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:31.010665 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:31.013028 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.013725 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.013728 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.013729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.013862 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.014224 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.014225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.014226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.014628 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:31.014699 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:31.015145 (+ 446us) server_negotiation.cc:299] Negotiation successful
0807 00:42:31.015307 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":87,"threads_started":1}
I20260807 00:42:31.016327 20091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:31.001277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41039 (local address 127.1.116.129:43705)
0807 00:42:31.001582 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:31.001642 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:31.002251 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:31.003035 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:31.003040 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:31.003406 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:31.003752 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:31.003759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.005581 (+ 1822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.005583 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:31.007061 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:31.007066 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.007992 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.007993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:31.008170 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:31.008896 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:31.008909 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:31.010447 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:31.013168 (+ 2721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:31.013180 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:31.013182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:31.013608 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:31.013964 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:31.013966 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:31.013966 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:31.014121 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:31.014760 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:31.014767 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:31.014933 (+ 166us) client_negotiation.cc:769] Sending connection context
0807 00:42:31.015843 (+ 910us) client_negotiation.cc:240] Negotiation successful
0807 00:42:31.016044 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":98,"threads_started":1}
I20260807 00:42:31.017375 20089 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41039
I20260807 00:42:31.017504 20089 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.017767 20089 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
W20260807 00:42:31.018934 20089 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:41039 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 00:42:31.019032 20089 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.019199 20089 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:31.019692 20089 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.019826 20089 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
W20260807 00:42:31.020258 20089 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:42:31.129611 20094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:31.129967 20094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:31.130012 20094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:31.131723 20094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:31.131842 20094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:31.131893 20094 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:42:31.133793 20094 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:42305
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=44937
--tserver_master_addrs=127.1.116.190:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:31.134136 20094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:31.134508 20094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:31.143747 20100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:31.145435 20105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:31.145777 20101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:31.146209 20094 server_base.cc:1034] running on GCE node
I20260807 00:42:31.146462 20094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:31.146862 20094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:31.148031 20094 hybrid_clock.cc:648] HybridClock initialized: now 1786063351148006 us; error 44 us; skew 500 ppm
Aug 07 00:42:31 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:31.151280 20094 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:42:31.151923 20094 webserver.cc:460] Webserver started at http://127.1.116.130:44937/ using document root <none> and password file <none>
I20260807 00:42:31.152247 20094 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:31.152341 20094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:42:31 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:31.166221 20094 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:31.170732 20116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:31.171025 20094 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.000s
I20260807 00:42:31.171152 20094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "c9f17aa8416a41299f2daf81c7bfe2a3"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "e578128462217c20325acd77c7433e96"
server_key_iv: "1afc257937257a542b853c92c35bfbc2"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:31.172648 20094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:31.204536 20094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:31.207278 20094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:31.207468 20094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:31.207826 20094 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:31.208410 20094 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:31.208475 20094 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:31.208510 20094 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:31.208529 20094 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:31.217217 20094 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:42305
I20260807 00:42:31.217454 20284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:42305 every 8 connection(s)
I20260807 00:42:31.217739 20094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:42:31.220569 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 20094
I20260807 00:42:31.220901 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.116.148:40595/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:41951
--local_ip_for_outbound_sockets=127.1.116.131
--tserver_master_addrs=127.1.116.190:41039
--webserver_port=35089
--webserver_interface=127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:42:31 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:31.230939 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:31.218712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:56917 (local address 127.1.116.190:41039)
0807 00:42:31.218805 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:31.218808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:31.219081 (+ 273us) server_negotiation.cc:407] Connection header received
0807 00:42:31.219607 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:31.219609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:31.219631 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:31.219745 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:31.220771 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.221529 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.223523 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.223865 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.223922 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:31.226985 (+ 3063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:31.226986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:31.226987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:31.226996 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:31.228862 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.229527 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.229529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.229530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.229619 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.230018 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.230019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.230019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.230162 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:31.230227 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:31.230692 (+ 465us) server_negotiation.cc:299] Negotiation successful
0807 00:42:31.230781 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:42:31.231220 20287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:31.218720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41039 (local address 127.1.116.130:56917)
0807 00:42:31.218906 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:31.218951 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:31.219479 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:31.219936 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:31.219944 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:31.220283 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:31.220640 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:31.220649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.221665 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.221667 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:31.223396 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:31.223400 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.224062 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.224063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:31.224237 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:31.225103 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:31.225116 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:31.226859 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:31.228973 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:31.228981 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:31.228984 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:31.229378 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:31.229700 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:31.229701 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:31.229702 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:31.229872 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:31.230247 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:31.230256 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:31.230470 (+ 214us) client_negotiation.cc:769] Sending connection context
0807 00:42:31.230648 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 00:42:31.230863 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":103,"threads_started":1}
I20260807 00:42:31.232096 20285 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41039
I20260807 00:42:31.232232 20285 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.232483 20285 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
W20260807 00:42:31.233577 20285 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:41039 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 00:42:31.233676 20285 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.233757 20285 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:31.234360 20285 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.234493 20285 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
W20260807 00:42:31.235347 20285 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:42:31.340725 20288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:31.340992 20288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:31.341035 20288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:31.342320 20288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:31.342392 20288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:31.342442 20288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:42:31.344100 20288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.116.148:40595/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:41951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=35089
--tserver_master_addrs=127.1.116.190:41039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:31.344400 20288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:31.344723 20288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:31.351423 20298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:31.351423 20297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:31.351562 20301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:31.351728 20288 server_base.cc:1034] running on GCE node
I20260807 00:42:31.352074 20288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:31.352432 20288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:31.353608 20288 hybrid_clock.cc:648] HybridClock initialized: now 1786063351353567 us; error 43 us; skew 500 ppm
Aug 07 00:42:31 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:31.356709 20288 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:42:31.357249 20288 webserver.cc:460] Webserver started at http://127.1.116.131:35089/ using document root <none> and password file <none>
I20260807 00:42:31.357512 20288 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:31.357604 20288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:42:31 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:31.376525 20288 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:31.377667 20314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:31.377852 20288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:31.377985 20288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "e3f1770567494386b807e1f887739892"
format_stamp: "Formatted at 2026-08-07 00:42:30 on dist-test-slave-h6n0"
server_key: "1c4255736c2006f7a91cdce66bcb7b6b"
server_key_iv: "52be0579c34328f29e6b9542030f9948"
server_key_version: "kuduclusterkey@0"
I20260807 00:42:31.378575 20288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:31.395181 20288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:31.397991 20288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:31.398213 20288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:31.398584 20288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:31.399039 20288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:31.399106 20288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:31.399154 20288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:31.399183 20288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:31.408159 20288 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:41951
I20260807 00:42:31.408272 20472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:41951 every 8 connection(s)
I20260807 00:42:31.408732 20288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:42:31.416908 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 20288
Aug 07 00:42:31 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063351, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:31.422395 20475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:31.409830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41039 (local address 127.1.116.131:35957)
0807 00:42:31.410046 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:31.410103 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:31.410859 (+ 756us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:31.411300 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:31.411305 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:31.411580 (+ 275us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:31.411913 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:31.411919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.413004 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.413005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:31.414386 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:31.414390 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.415030 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.415031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:31.415200 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:31.416006 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:31.416018 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:31.418103 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:31.420415 (+ 2312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:31.420424 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:31.420427 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:31.420785 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:31.421028 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:31.421029 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:31.421029 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:31.421200 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:31.421551 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:31.421558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:31.421773 (+ 215us) client_negotiation.cc:769] Sending connection context
0807 00:42:31.421899 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 00:42:31.422084 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":132,"threads_started":1}
I20260807 00:42:31.422951 20093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:31.409822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:35957 (local address 127.1.116.190:41039)
0807 00:42:31.409887 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:31.409890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:31.410212 (+ 322us) server_negotiation.cc:407] Connection header received
0807 00:42:31.411008 (+ 796us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:31.411010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:31.411026 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:31.411061 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:31.412044 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.412867 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.414498 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.414844 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.414897 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:31.418229 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:31.418231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:31.418232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:31.418241 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:31.420315 (+ 2074us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.420880 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.420882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.420883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.420954 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.421306 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.421307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.421307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.421461 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:31.421523 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:31.422756 (+ 1233us) server_negotiation.cc:299] Negotiation successful
0807 00:42:31.422854 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 00:42:31.423475 20473 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41039
I20260807 00:42:31.423592 20473 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.423839 20473 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
W20260807 00:42:31.424660 20473 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:41039 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 00:42:31.424757 20473 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.424834 20473 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:31.425143 20473 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:31.425220 20473 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
W20260807 00:42:31.425503 20473 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:42:31.430716 20481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:31.417638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56148 (local address 127.1.116.190:41039)
0807 00:42:31.418640 (+ 1002us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:31.418642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:31.419288 (+ 646us) server_negotiation.cc:407] Connection header received
0807 00:42:31.419432 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:31.419433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:31.419477 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:31.419549 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:31.421338 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.422171 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.424970 (+ 2799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:31.425734 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:31.425822 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:31.427352 (+ 1530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:31.427354 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:31.427354 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:31.427362 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:31.428975 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.429511 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.429514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.429514 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.429633 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:31.429901 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:31.429902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:31.429902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:31.430100 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:31.430352 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:31.430425 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 00:42:31.430587 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969,"thread_start_us":86,"threads_started":1}
Aug 07 00:42:32 dist-test-slave-h6n0 krb5kdc[18439](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786063352, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:32.020484 20089 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:32.020632 20089 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:32.021114 19805 ts_manager.cc:194] Registered new tserver with Master: 9f5f66be4a1747e4a1fcdddc05424f9d (127.1.116.129:33175)
I20260807 00:42:32.022153 19805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:43705
I20260807 00:42:32.235574 20285 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:32.235805 20285 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:32.236189 19805 ts_manager.cc:194] Registered new tserver with Master: c9f17aa8416a41299f2daf81c7bfe2a3 (127.1.116.130:42305)
I20260807 00:42:32.236745 19805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:56917
Aug 07 00:42:32 dist-test-slave-h6n0 krb5kdc[18439](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063352, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for HTTP/127.1.116.148@KRBTEST.COM
I20260807 00:42:32.425685 20473 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:32.425839 20473 heartbeater.cc:511] Master 127.1.116.190:41039 requested a full tablet report, sending...
I20260807 00:42:32.426126 19805 ts_manager.cc:194] Registered new tserver with Master: e3f1770567494386b807e1f887739892 (127.1.116.131:41951)
I20260807 00:42:32.426810 19805 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:35957
I20260807 00:42:32.428246 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:42:32.442806 20500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.429269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.1.116.190:41039)
0807 00:42:32.430042 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:32.430045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:32.430170 (+ 125us) server_negotiation.cc:407] Connection header received
0807 00:42:32.430301 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:32.430303 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:32.430319 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:32.430368 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:32.432935 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.433647 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.435359 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.435662 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.435715 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.437829 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:32.437830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:32.437830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:32.437838 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:32.441207 (+ 3369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:32.441671 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:32.441672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:32.441672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:32.441740 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:32.442005 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:32.442006 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:32.442006 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:32.442162 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:32.442245 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:32.442597 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 00:42:32.442689 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":77,"threads_started":1}
I20260807 00:42:32.445595 19805 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:42:32.449615 19805 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:42:32.462359 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.455862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33175 (local address 127.0.0.1:44972)
0807 00:42:32.457176 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:32.457186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:32.457230 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:32.457653 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:32.457656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:32.457658 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:32.457723 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.457725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.459402 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.459403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.460656 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.460659 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.462033 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.462034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.462116 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.462128 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 00:42:32.462200 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:42:32.462250 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"thread_start_us":159,"threads_started":1}
I20260807 00:42:32.465050 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.456267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44972 (local address 127.1.116.129:33175)
0807 00:42:32.457366 (+ 1099us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:32.457371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:32.457383 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:42:32.457422 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:32.457424 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:32.457443 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:32.457532 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:32.457832 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.459277 (+ 1445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.461057 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.461842 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.461931 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.461992 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:32.464930 (+ 2938us) server_negotiation.cc:299] Negotiation successful
0807 00:42:32.464976 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1046,"thread_start_us":92,"threads_started":1}
I20260807 00:42:32.466162 19997 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2afc57261f417dbb4a0178c3eb29e3 (DEFAULT_TABLE table=test-table [id=ec0a3b86069a4afe9f19b8c91eacbf14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:32.466712 19997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2afc57261f417dbb4a0178c3eb29e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:32.469491 20511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.455579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:41951 (local address 127.0.0.1:54502)
0807 00:42:32.456935 (+ 1356us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:32.456976 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:32.457046 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:32.465641 (+ 8595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:32.465642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:32.465645 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:32.465690 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.465692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.467194 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.467195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.468316 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.468320 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.469273 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.469274 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.469334 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.469339 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:42:32.469371 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 00:42:32.469409 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1326,"thread_start_us":109,"threads_started":1}
I20260807 00:42:32.469935 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.455668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54502 (local address 127.1.116.131:41951)
0807 00:42:32.465346 (+ 9678us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:32.465351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:32.465363 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:42:32.465399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:32.465401 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:32.465421 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:32.465504 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:32.465799 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.466833 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.468424 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.469167 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.469729 (+ 562us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.469797 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:32.469825 (+ 28us) server_negotiation.cc:299] Negotiation successful
0807 00:42:32.469864 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9642,"thread_start_us":59,"threads_started":1}
I20260807 00:42:32.470721 20357 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2afc57261f417dbb4a0178c3eb29e3 (DEFAULT_TABLE table=test-table [id=ec0a3b86069a4afe9f19b8c91eacbf14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:32.471122 20357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2afc57261f417dbb4a0178c3eb29e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:32.471736 20522 tablet_bootstrap.cc:492] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d: Bootstrap starting.
I20260807 00:42:32.472443 20522 tablet_bootstrap.cc:654] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:32.472622 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.456586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34930 (local address 127.1.116.130:42305)
0807 00:42:32.456854 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:32.456860 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:32.462937 (+ 6077us) server_negotiation.cc:407] Connection header received
0807 00:42:32.462984 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:32.462987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:32.463005 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:32.463091 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:32.463306 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.464802 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.471424 (+ 6622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.472150 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.472212 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.472263 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:32.472486 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 00:42:32.472546 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":86,"threads_started":1}
I20260807 00:42:32.472843 20522 log.cc:824] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:32.472922 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.456374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:42305 (local address 127.0.0.1:34930)
0807 00:42:32.462802 (+ 6428us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:32.462813 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:32.462858 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:32.463191 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:32.463192 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:32.463194 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:32.463225 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.463226 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.464953 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.464955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.471035 (+ 6080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.471039 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.472328 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.472329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.472390 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.472392 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:42:32.472719 (+ 327us) client_negotiation.cc:240] Negotiation successful
0807 00:42:32.472851 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6415,"thread_start_us":85,"threads_started":1}
I20260807 00:42:32.474190 20522 tablet_bootstrap.cc:492] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d: No bootstrap required, opened a new log
I20260807 00:42:32.474263 20522 ts_tablet_manager.cc:1397] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:42:32.474670 20178 tablet_service.cc:1467] Processing CreateTablet for tablet 8b2afc57261f417dbb4a0178c3eb29e3 (DEFAULT_TABLE table=test-table [id=ec0a3b86069a4afe9f19b8c91eacbf14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:32.475013 20178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8b2afc57261f417dbb4a0178c3eb29e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:32.476925 20522 raft_consensus.cc:348] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.477402 20522 raft_consensus.cc:374] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:32.477491 20522 raft_consensus.cc:729] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f5f66be4a1747e4a1fcdddc05424f9d, State: Initialized, Role: FOLLOWER
I20260807 00:42:32.477702 20522 consensus_queue.cc:260] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.478129 20522 ts_tablet_manager.cc:1428] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260807 00:42:32.478335 20089 heartbeater.cc:503] Master 127.1.116.190:41039 was elected leader, sending a full tablet report...
I20260807 00:42:32.480137 20527 tablet_bootstrap.cc:492] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Bootstrap starting.
I20260807 00:42:32.480396 20526 tablet_bootstrap.cc:492] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Bootstrap starting.
I20260807 00:42:32.480942 20527 tablet_bootstrap.cc:654] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:32.481186 20526 tablet_bootstrap.cc:654] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:32.481359 20527 log.cc:824] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:32.481652 20526 log.cc:824] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:32.482208 20527 tablet_bootstrap.cc:492] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: No bootstrap required, opened a new log
I20260807 00:42:32.482286 20527 ts_tablet_manager.cc:1397] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:42:32.484692 20527 raft_consensus.cc:348] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.484869 20527 raft_consensus.cc:374] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:32.484902 20527 raft_consensus.cc:729] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3f1770567494386b807e1f887739892, State: Initialized, Role: FOLLOWER
I20260807 00:42:32.485059 20527 consensus_queue.cc:260] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.485330 20527 ts_tablet_manager.cc:1428] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:42:32.485452 20473 heartbeater.cc:503] Master 127.1.116.190:41039 was elected leader, sending a full tablet report...
I20260807 00:42:32.487471 20531 raft_consensus.cc:482] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:32.487561 20531 raft_consensus.cc:504] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.487911 20531 leader_election.cc:288] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9f17aa8416a41299f2daf81c7bfe2a3 (127.1.116.130:42305), 9f5f66be4a1747e4a1fcdddc05424f9d (127.1.116.129:33175)
I20260807 00:42:32.492240 20514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.488353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:34679 (local address 127.1.116.129:33175)
0807 00:42:32.488395 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:32.488397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:32.488407 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:42:32.488452 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:32.488452 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:32.488462 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:32.488519 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:32.488898 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.489677 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.491168 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.491783 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.491864 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.491874 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:32.492127 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 00:42:32.492172 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 00:42:32.492801 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.488118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33175 (local address 127.1.116.131:34679)
0807 00:42:32.488295 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:32.488311 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:32.488383 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:32.488539 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:32.488540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:32.488543 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:32.488799 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.488803 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.489782 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.489783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.491064 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.491067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.491944 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.491945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.492031 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.492036 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:42:32.492541 (+ 505us) client_negotiation.cc:240] Negotiation successful
0807 00:42:32.492658 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":40,"thread_start_us":91,"threads_started":1}
I20260807 00:42:32.493250 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2afc57261f417dbb4a0178c3eb29e3" candidate_uuid: "e3f1770567494386b807e1f887739892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" is_pre_election: true
I20260807 00:42:32.493413 20020 raft_consensus.cc:2454] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e3f1770567494386b807e1f887739892 in term 0.
I20260807 00:42:32.493711 20318 leader_election.cc:302] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f5f66be4a1747e4a1fcdddc05424f9d, e3f1770567494386b807e1f887739892; no voters:
I20260807 00:42:32.493851 20531 raft_consensus.cc:2781] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:42:32.493894 20531 raft_consensus.cc:482] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:42:32.493916 20531 raft_consensus.cc:3037] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:32.494822 20516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.488736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:55073 (local address 127.1.116.130:42305)
0807 00:42:32.488785 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:32.488787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:32.491260 (+ 2473us) server_negotiation.cc:407] Connection header received
0807 00:42:32.491357 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:32.491358 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:32.491368 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:32.491397 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:32.491602 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.492321 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.493828 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.494370 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.494415 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.494425 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:32.494706 (+ 281us) server_negotiation.cc:299] Negotiation successful
0807 00:42:32.494749 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:42:32.494807 20531 raft_consensus.cc:504] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.494952 20531 leader_election.cc:288] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [CANDIDATE]: Term 1 election: Requested vote from peers c9f17aa8416a41299f2daf81c7bfe2a3 (127.1.116.130:42305), 9f5f66be4a1747e4a1fcdddc05424f9d (127.1.116.129:33175)
I20260807 00:42:32.495060 20534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.488665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:42305 (local address 127.1.116.131:55073)
0807 00:42:32.491196 (+ 2531us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:32.491207 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:32.491288 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:32.491455 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:32.491455 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:32.491457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:32.491536 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.491537 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.492417 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.492419 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.493721 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:32.493724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.494483 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.494483 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:32.494593 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.494599 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:42:32.494931 (+ 332us) client_negotiation.cc:240] Negotiation successful
0807 00:42:32.494979 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2513,"thread_start_us":77,"threads_started":1}
I20260807 00:42:32.495424 20215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2afc57261f417dbb4a0178c3eb29e3" candidate_uuid: "e3f1770567494386b807e1f887739892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" is_pre_election: true
I20260807 00:42:32.497021 20219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2afc57261f417dbb4a0178c3eb29e3" candidate_uuid: "e3f1770567494386b807e1f887739892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3"
W20260807 00:42:32.497164 20319 leader_election.cc:341] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c9f17aa8416a41299f2daf81c7bfe2a3 (127.1.116.130:42305): Illegal state: must be running to vote when last-logged opid is not known
W20260807 00:42:32.497299 20319 leader_election.cc:341] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c9f17aa8416a41299f2daf81c7bfe2a3 (127.1.116.130:42305): Illegal state: must be running to vote when last-logged opid is not known
I20260807 00:42:32.497359 20020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8b2afc57261f417dbb4a0178c3eb29e3" candidate_uuid: "e3f1770567494386b807e1f887739892" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d"
I20260807 00:42:32.497442 20020 raft_consensus.cc:3037] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:32.498382 20020 raft_consensus.cc:2454] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e3f1770567494386b807e1f887739892 in term 1.
I20260807 00:42:32.498580 20318 leader_election.cc:302] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9f5f66be4a1747e4a1fcdddc05424f9d, e3f1770567494386b807e1f887739892; no voters: c9f17aa8416a41299f2daf81c7bfe2a3
I20260807 00:42:32.498692 20531 raft_consensus.cc:2781] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:32.498739 20531 raft_consensus.cc:686] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 1 LEADER]: Becoming Leader. State: Replica: e3f1770567494386b807e1f887739892, State: Running, Role: LEADER
I20260807 00:42:32.498821 20531 consensus_queue.cc:237] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.499559 19805 catalog_manager.cc:5495] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 reported cstate change: term changed from 0 to 1, leader changed from <none> to e3f1770567494386b807e1f887739892 (127.1.116.131). New cstate: current_term: 1 leader_uuid: "e3f1770567494386b807e1f887739892" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } health_report { overall_health: UNKNOWN } } }
I20260807 00:42:32.501080 20526 tablet_bootstrap.cc:492] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: No bootstrap required, opened a new log
I20260807 00:42:32.501159 20526 ts_tablet_manager.cc:1397] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.000s
W20260807 00:42:32.501780 20090 tablet.cc:2369] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:42:32.502846 20526 raft_consensus.cc:348] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.502947 20526 raft_consensus.cc:374] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:32.502969 20526 raft_consensus.cc:729] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9f17aa8416a41299f2daf81c7bfe2a3, State: Initialized, Role: FOLLOWER
I20260807 00:42:32.503082 20526 consensus_queue.cc:260] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3f1770567494386b807e1f887739892" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41951 } } peers { permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 } } peers { permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 } }
I20260807 00:42:32.503348 20526 ts_tablet_manager.cc:1428] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:42:32.503468 20285 heartbeater.cc:503] Master 127.1.116.190:41039 was elected leader, sending a full tablet report...
I20260807 00:42:32.516700 20510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.514263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54516 (local address 127.1.116.131:41951)
0807 00:42:32.514304 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:32.514306 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:32.514352 (+ 46us) server_negotiation.cc:407] Connection header received
0807 00:42:32.514429 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:32.514429 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:32.514446 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:32.514513 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:42:32.514705 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.515171 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.515928 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:32.516146 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:32.516224 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:32.516380 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:42:32.516474 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:42:32.516565 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:32.516603 (+ 38us) server_negotiation.cc:299] Negotiation successful
0807 00:42:32.516622 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:42:32.518752 20020 raft_consensus.cc:1261] T 8b2afc57261f417dbb4a0178c3eb29e3 P 9f5f66be4a1747e4a1fcdddc05424f9d [term 1 FOLLOWER]: Refusing update from remote peer e3f1770567494386b807e1f887739892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:32.519016 20219 raft_consensus.cc:3037] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:32.519716 20219 raft_consensus.cc:1261] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [term 1 FOLLOWER]: Refusing update from remote peer e3f1770567494386b807e1f887739892: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:32.519869 20531 consensus_queue.cc:1035] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f5f66be4a1747e4a1fcdddc05424f9d" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:32.520018 20530 consensus_queue.cc:1035] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9f17aa8416a41299f2daf81c7bfe2a3" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:32.524557 20542 mvcc.cc:205] Tried to move back new op lower bound from 7315715491914375168 to 7315715491835625472. Current Snapshot: MvccSnapshot[applied={T|T < 7315715491914375168}]
I20260807 00:42:32.527626 19805 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:32.527722 19805 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:32.528421 19805 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 01e7f4df5cc14ac69089c7f94ff325be: Sending DeleteTablet for 3 replicas of tablet 8b2afc57261f417dbb4a0178c3eb29e3
I20260807 00:42:32.528883 20357 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2afc57261f417dbb4a0178c3eb29e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:32 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:54502
I20260807 00:42:32.529127 19997 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2afc57261f417dbb4a0178c3eb29e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:32 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:44972
I20260807 00:42:32.529497 20178 tablet_service.cc:1514] Processing DeleteTablet for tablet 8b2afc57261f417dbb4a0178c3eb29e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:32 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:34930
I20260807 00:42:32.529989 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 19866
I20260807 00:42:32.542667 20552 tablet_replica.cc:330] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: stopping tablet replica
I20260807 00:42:32.542848 20552 raft_consensus.cc:2229] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:42:32.543016 20552 raft_consensus.cc:2258] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:32.543640 20552 ts_tablet_manager.cc:1905] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:32.545482 20552 ts_tablet_manager.cc:1918] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:32.545570 20552 log.cc:1196] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/8b2afc57261f417dbb4a0178c3eb29e3
I20260807 00:42:32.545866 20552 ts_tablet_manager.cc:1939] T 8b2afc57261f417dbb4a0178c3eb29e3 P e3f1770567494386b807e1f887739892: Deleting consensus metadata
I20260807 00:42:32.546988 20557 tablet_replica.cc:330] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: stopping tablet replica
I20260807 00:42:32.546993 19783 catalog_manager.cc:4841] TS e3f1770567494386b807e1f887739892 (127.1.116.131:41951): tablet 8b2afc57261f417dbb4a0178c3eb29e3 (table test-table [id=ec0a3b86069a4afe9f19b8c91eacbf14]) successfully deleted
I20260807 00:42:32.547137 20557 raft_consensus.cc:2229] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:42:32.547277 20557 raft_consensus.cc:2258] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:32.548293 20557 ts_tablet_manager.cc:1905] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:32.549990 20557 ts_tablet_manager.cc:1918] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:32.550074 20557 log.cc:1196] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/8b2afc57261f417dbb4a0178c3eb29e3
I20260807 00:42:32.550380 20557 ts_tablet_manager.cc:1939] T 8b2afc57261f417dbb4a0178c3eb29e3 P c9f17aa8416a41299f2daf81c7bfe2a3: Deleting consensus metadata
I20260807 00:42:32.550926 19787 catalog_manager.cc:4841] TS c9f17aa8416a41299f2daf81c7bfe2a3 (127.1.116.130:42305): tablet 8b2afc57261f417dbb4a0178c3eb29e3 (table test-table [id=ec0a3b86069a4afe9f19b8c91eacbf14]) successfully deleted
W20260807 00:42:32.554179 19785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:33175 (error 108)
I20260807 00:42:32.554915 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 20094
I20260807 00:42:32.613160 20508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:32.612967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33175 (local address 127.0.0.1:44978)
0807 00:42:32.613025 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:32.613101 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:33175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 00:42:32.618546 19785 catalog_manager.cc:4584] TS 9f5f66be4a1747e4a1fcdddc05424f9d (127.1.116.129:33175): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8b2afc57261f417dbb4a0178c3eb29e3: Network error: Client connection negotiation failed: client connection to 127.1.116.129:33175: connect: Connection refused (error 111)
I20260807 00:42:32.620249 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 20288
I20260807 00:42:32.634948 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 19762
I20260807 00:42:32.648910 1490 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 00:42:32.993929 1490 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 00:42:32.994073 1490 mini_ranger.cc:66] Stopping Ranger...
I20260807 00:42:33.349208 1490 mini_ranger.cc:68] Stopped Ranger
2026-08-07 00:42:33.349 UTC [18483] LOG: received smart shutdown request
2026-08-07 00:42:33.353 UTC [18483] LOG: background worker "logical replication launcher" (PID 18494) exited with exit code 1
2026-08-07 00:42:33.353 UTC [18487] LOG: shutting down
2026-08-07 00:42:33.511 UTC [18483] LOG: database system is shut down
2026-08-07T00:42:33Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52134 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:42:33 dist-test-slave-h6n0 krb5kdc[20571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:42:33 dist-test-slave-h6n0 krb5kdc[20571](info): set up 2 sockets
Aug 07 00:42:33 dist-test-slave-h6n0 krb5kdc[20571](info): commencing operation
krb5kdc: starting...
W20260807 00:42:34.876288 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.211s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:42:34 dist-test-slave-h6n0 krb5kdc[20571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:42:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:42:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:42:35.010798 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:46181
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:46181,127.1.116.189:39981,127.1.116.188:34939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:35.096138 20590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:35.096362 20590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:35.096401 20590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:35.097517 20590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:35.097582 20590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:35.097607 20590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:35.097630 20590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:35.097657 20590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:35.099186 20590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:46181,127.1.116.189:39981,127.1.116.188:34939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:46181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:35.099457 20590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:35.099740 20590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:35.105255 20597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:35.105194 20596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:35.105149 20601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:35.105242 20590 server_base.cc:1034] running on GCE node
I20260807 00:42:35.105746 20590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:35.106112 20590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:35.107328 20590 hybrid_clock.cc:648] HybridClock initialized: now 1786063355107303 us; error 59 us; skew 500 ppm
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:35.114720 20590 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:35.115247 20590 webserver.cc:460] Webserver started at http://127.1.116.190:40705/ using document root <none> and password file <none>
I20260807 00:42:35.115526 20590 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:35.115597 20590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:35.115767 20590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:35.117050 20590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "4dbb3c140c054bb596e98d083665c2fd"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "a92d3af647f80918096623831deae416"
server_key_iv: "39172e064085e1c50572bafe5ca013ed"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.117442 20590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "4dbb3c140c054bb596e98d083665c2fd"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "a92d3af647f80918096623831deae416"
server_key_iv: "39172e064085e1c50572bafe5ca013ed"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.118969 20590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:42:35.119828 20609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:35.120061 20590 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:35.120219 20590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "4dbb3c140c054bb596e98d083665c2fd"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "a92d3af647f80918096623831deae416"
server_key_iv: "39172e064085e1c50572bafe5ca013ed"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.120303 20590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:35.144577 20590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:35.147254 20590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:35.147449 20590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:35.153237 20590 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:46181
I20260807 00:42:35.153307 20682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:46181 every 8 connection(s)
I20260807 00:42:35.153776 20590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:35.154865 20683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:35.156489 20683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.157469 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 20590
I20260807 00:42:35.157598 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:42:35.157807 1490 external_mini_cluster.cc:1428] Setting key 830710dc6dd22332234c09a937c0ce3c
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.189@KRBTEST.COM; defaulting to no policy
I20260807 00:42:35.168440 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.157275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58966 (local address 127.1.116.190:46181)
0807 00:42:35.157497 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.157509 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.157692 (+ 183us) server_negotiation.cc:407] Connection header received
0807 00:42:35.158482 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.158485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.158514 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.158658 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.159336 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.160379 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.161279 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.161754 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.161877 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.164367 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.164376 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.164377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.164386 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.166611 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.167041 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.167044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.167045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.167155 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.167413 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.167414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.167415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.167711 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.167818 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.168106 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.168275 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260807 00:42:35.168449 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.157236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:46181 (local address 127.0.0.1:58966)
0807 00:42:35.157485 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.157553 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.158288 (+ 735us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.158694 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.158697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.158987 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.159214 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.159218 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.160490 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.160492 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.161178 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.161180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.161858 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.161859 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.161951 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.162571 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.162583 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.164258 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.166707 (+ 2449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.166714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.166716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.166937 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.167244 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.167244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.167245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.167333 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.167849 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.167859 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.167944 (+ 85us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.168040 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.168212 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":112,"threads_started":1}
I20260807 00:42:35.169272 20683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.169678 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.169528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35468)
0807 00:42:35.169599 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.169653 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 00:42:35.169881 20614 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
I20260807 00:42:35.170490 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.170372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35476)
0807 00:42:35.170430 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.170468 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.189@KRBTEST.COM" created.
W20260807 00:42:35.170637 20683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:39981: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab.
Entry for principal kudu/127.1.116.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab.
I20260807 00:42:35.185062 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:39981
--webserver_interface=127.1.116.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab
--principal=kudu/127.1.116.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:46181,127.1.116.189:39981,127.1.116.188:34939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:42:35.219894 20683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } attempt: 1
I20260807 00:42:35.220561 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.220385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35482)
0807 00:42:35.220465 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.220525 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 00:42:35.220985 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.220887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35490)
0807 00:42:35.220933 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.220967 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 00:42:35.221081 20683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:39981: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
W20260807 00:42:35.269752 20691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:35.269977 20691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:35.270015 20691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:35.271245 20691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:35.271306 20691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:35.271332 20691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:35.271363 20691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:35.271387 20691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:35.273067 20691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:46181,127.1.116.189:39981,127.1.116.188:34939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab
--principal=kudu/127.1.116.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:39981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.116.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:35.273350 20691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:35.273583 20691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:35.278939 20698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:35.278949 20699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:35.279163 20691 server_base.cc:1034] running on GCE node
W20260807 00:42:35.278949 20702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:35.279474 20691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:35.279798 20691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:35.280966 20691 hybrid_clock.cc:648] HybridClock initialized: now 1786063355280940 us; error 46 us; skew 500 ppm
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:35.283551 20691 init.cc:377] Logged in from keytab as kudu/127.1.116.189@KRBTEST.COM (short username kudu)
I20260807 00:42:35.284035 20691 webserver.cc:460] Webserver started at http://127.1.116.189:46199/ using document root <none> and password file <none>
I20260807 00:42:35.284283 20691 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:35.284353 20691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:35.284515 20691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:35.285523 20691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data/instance:
uuid: "095e644e1b7946f68bd5e752576c5788"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "0c05980f030c0d7b43bddea35c3103ff"
server_key_iv: "76d2cc7e2e098d211201f72d1cb1fab9"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.285879 20691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/wal/instance:
uuid: "095e644e1b7946f68bd5e752576c5788"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "0c05980f030c0d7b43bddea35c3103ff"
server_key_iv: "76d2cc7e2e098d211201f72d1cb1fab9"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.287369 20691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:42:35.288197 20713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:35.288434 20691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:35.288549 20691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/wal
uuid: "095e644e1b7946f68bd5e752576c5788"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "0c05980f030c0d7b43bddea35c3103ff"
server_key_iv: "76d2cc7e2e098d211201f72d1cb1fab9"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.288623 20691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:35.289220 20683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } attempt: 2
I20260807 00:42:35.289695 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.289546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35500)
0807 00:42:35.289610 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.289654 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 00:42:35.290040 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.289949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35516)
0807 00:42:35.289991 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.290015 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260807 00:42:35.290131 20683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:39981: Network error: Client connection negotiation failed: client connection to 127.1.116.189:39981: connect: Connection refused (error 111)
I20260807 00:42:35.310071 20691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:35.312058 20691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:35.312219 20691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:35.316771 20691 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.189:39981
I20260807 00:42:35.316843 20786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.189:39981 every 8 connection(s)
I20260807 00:42:35.317200 20691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
I20260807 00:42:35.318260 20787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:35.320272 20787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.321422 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 20691
I20260807 00:42:35.321554 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-1/wal/instance
I20260807 00:42:35.321748 1490 external_mini_cluster.cc:1428] Setting key 262fb225292627516997f489761b29d5
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.188@KRBTEST.COM; defaulting to no policy
I20260807 00:42:35.331614 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.321061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58970 (local address 127.1.116.190:46181)
0807 00:42:35.321133 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.321136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.321276 (+ 140us) server_negotiation.cc:407] Connection header received
0807 00:42:35.322192 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.322193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.322209 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.322267 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.323197 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.323868 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.325056 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.325280 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.325359 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.328061 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.328063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.328063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.328088 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.329858 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.330442 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.330443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.330444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.330513 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.330823 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.330824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.330825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.330998 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.331072 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.331426 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.331517 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:42:35.331693 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.320980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:46181 (local address 127.0.0.1:58970)
0807 00:42:35.321153 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.321195 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.322044 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.322367 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.322372 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.322796 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.323089 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.323096 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.323988 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.323990 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.324951 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.324955 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.325369 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.325370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.325520 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.326089 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.326099 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.327962 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.329988 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.329996 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.329999 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.330352 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.330604 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.330606 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.330606 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.330728 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.331089 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.331097 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.331239 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.331360 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.331533 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":96,"threads_started":1}
I20260807 00:42:35.332507 20787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.188@KRBTEST.COM" created.
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:42:35.340935 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.332835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35532)
0807 00:42:35.332906 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.332916 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.332957 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.333241 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.333242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.333394 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.333448 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.333450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.334322 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.334323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.334908 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.334910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.335538 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.335540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.335683 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.336507 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.336513 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.337970 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.339839 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.339840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.339841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.340020 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.340262 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.340262 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.340263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.340319 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.340612 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.340612 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.340651 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.340719 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.340825 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 00:42:35.341023 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.332899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.1.116.189:39981)
0807 00:42:35.333100 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.333103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.333126 (+ 23us) server_negotiation.cc:407] Connection header received
0807 00:42:35.333144 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.333145 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.333156 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.333241 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.333533 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.334229 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.335012 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.335421 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.335508 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.338074 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.338082 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.338082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.338090 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.339730 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.340118 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.340121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.340121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.340181 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.340397 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.340397 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.340398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.340533 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.340619 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.340752 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.340879 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":171,"thread_start_us":111,"threads_started":1}
I20260807 00:42:35.341842 20787 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.342198 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.342076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45596)
0807 00:42:35.342126 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.342170 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 00:42:35.342332 20717 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
I20260807 00:42:35.342856 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.342761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45606)
0807 00:42:35.342810 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.342834 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 00:42:35.342979 20787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:34939: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab.
Entry for principal kudu/127.1.116.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab.
I20260807 00:42:35.350857 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:34939
--webserver_interface=127.1.116.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab
--principal=kudu/127.1.116.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:46671
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:46181,127.1.116.189:39981,127.1.116.188:34939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:42:35.381304 20683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } attempt: 3
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:42:35.389070 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.381889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35546 (local address 127.1.116.189:39981)
0807 00:42:35.381982 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.381984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.381994 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:42:35.382029 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.382030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.382041 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.382110 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.382459 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.383042 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.383752 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.383956 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.384070 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.386367 (+ 2297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.386369 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.386369 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.386375 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.387870 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.388263 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.388265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.388265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.388325 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.388528 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.388529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.388530 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.388651 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.388726 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.388891 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.388969 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:42:35.389081 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.381792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35546)
0807 00:42:35.381863 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.381873 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.381944 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.382118 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.382119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.382300 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.382377 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.382378 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.383130 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.383131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.383653 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.383655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.384069 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.384070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.384154 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.384628 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.384636 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.386244 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.387975 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.387977 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.387977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.388171 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.388395 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.388396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.388397 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.388461 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.388730 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.388731 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.388778 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.388865 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.388967 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260807 00:42:35.389770 20683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.390235 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.390110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45612)
0807 00:42:35.390179 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.390210 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 00:42:35.390693 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.390613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45618)
0807 00:42:35.390654 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.390680 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 00:42:35.390816 20683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:34939: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
I20260807 00:42:35.392159 20787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } attempt: 1
I20260807 00:42:35.392596 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.392493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45632)
0807 00:42:35.392545 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.392570 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 00:42:35.393000 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.392929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45638)
0807 00:42:35.392961 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.392984 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 00:42:35.393124 20787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:34939: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
I20260807 00:42:35.422000 20683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } attempt: 1
I20260807 00:42:35.422859 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.422633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45644)
0807 00:42:35.422767 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.422824 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 00:42:35.423313 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.423192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45648)
0807 00:42:35.423267 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.423294 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260807 00:42:35.423548 20683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:34939: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
W20260807 00:42:35.447664 20795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:35.447921 20795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:35.447944 20795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:35.449180 20795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:35.449231 20795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:35.449247 20795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:35.449260 20795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:35.449272 20795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:35.450954 20795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:46671
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:46181,127.1.116.189:39981,127.1.116.188:34939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab
--principal=kudu/127.1.116.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:34939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.116.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:35.451267 20795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:35.451558 20795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:35.457219 20800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:35.457208 20801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:35.457208 20808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:35.457727 20795 server_base.cc:1034] running on GCE node
I20260807 00:42:35.458020 20795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:35.458316 20795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:35.459471 20795 hybrid_clock.cc:648] HybridClock initialized: now 1786063355459428 us; error 51 us; skew 500 ppm
I20260807 00:42:35.461278 20787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } attempt: 2
I20260807 00:42:35.461896 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.461700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45662)
0807 00:42:35.461810 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.461855 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:35.462386 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.462213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45668)
0807 00:42:35.462301 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.462351 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 00:42:35.462563 20787 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:34939: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
I20260807 00:42:35.462718 20795 init.cc:377] Logged in from keytab as kudu/127.1.116.188@KRBTEST.COM (short username kudu)
I20260807 00:42:35.463280 20795 webserver.cc:460] Webserver started at http://127.1.116.188:36811/ using document root <none> and password file <none>
I20260807 00:42:35.463572 20795 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:35.463630 20795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:35.463795 20795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:35.465061 20795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data/instance:
uuid: "4acdef11228f4deeba9cfbe9d2827952"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "6f64cdb85af159fd419dfd53e0af939a"
server_key_iv: "6e919b4c3a0b5d7e7277e6281090e676"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.465483 20795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/wal/instance:
uuid: "4acdef11228f4deeba9cfbe9d2827952"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "6f64cdb85af159fd419dfd53e0af939a"
server_key_iv: "6e919b4c3a0b5d7e7277e6281090e676"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.467116 20795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:42:35.468144 20817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:35.468454 20795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:35.468576 20795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/wal
uuid: "4acdef11228f4deeba9cfbe9d2827952"
format_stamp: "Formatted at 2026-08-07 00:42:35 on dist-test-slave-h6n0"
server_key: "6f64cdb85af159fd419dfd53e0af939a"
server_key_iv: "6e919b4c3a0b5d7e7277e6281090e676"
server_key_version: "encryptionkey@0"
I20260807 00:42:35.468655 20795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:35.494855 20795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:35.497112 20795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:35.497309 20795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:35.498716 20683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } attempt: 2
I20260807 00:42:35.499208 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.499061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45684)
0807 00:42:35.499135 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.499179 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 00:42:35.499567 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.499462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45694)
0807 00:42:35.499512 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.499538 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 00:42:35.499660 20683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:34939: Network error: Client connection negotiation failed: client connection to 127.1.116.188:34939: connect: Connection refused (error 111)
I20260807 00:42:35.502920 20795 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.188:34939
I20260807 00:42:35.502985 20895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.188:34939 every 8 connection(s)
I20260807 00:42:35.503384 20795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
I20260807 00:42:35.504506 20896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:35.506132 20896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.508309 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 20795
I20260807 00:42:35.508452 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786063190385254-1490-0/minicluster-data/master-2/wal/instance
I20260807 00:42:35.508661 1490 external_mini_cluster.cc:1428] Setting key 454ee79270db73d76bb7d779ca85b9b0
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:42:35.517185 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.506877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58986 (local address 127.1.116.190:46181)
0807 00:42:35.506926 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.506927 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.507181 (+ 254us) server_negotiation.cc:407] Connection header received
0807 00:42:35.507701 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.507703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.507717 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.507784 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.508441 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.509096 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.510643 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.510852 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.510929 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.513720 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.513721 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.513722 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.513731 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.515419 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.515927 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.515928 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.515929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.515992 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.516353 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.516354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.516355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.516526 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.516610 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.516972 (+ 362us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.517068 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:42:35.517311 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.506862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:46181 (local address 127.0.0.1:58986)
0807 00:42:35.507040 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.507078 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.507588 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.507838 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.507841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.508136 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.508331 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.508335 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.509187 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.509189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.510482 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.510486 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.510950 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.510952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.511155 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.511812 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.511823 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.513613 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.515515 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.515523 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.515526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.515844 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.516104 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.516106 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.516106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.516266 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.516618 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.516624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.516786 (+ 162us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.516908 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.517112 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260807 00:42:35.518052 20896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.519467 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.509129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45704 (local address 127.1.116.188:34939)
0807 00:42:35.509319 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.509323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.509367 (+ 44us) server_negotiation.cc:407] Connection header received
0807 00:42:35.509460 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.509461 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.509480 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.509554 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.510301 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.511168 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.512313 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.512756 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.512845 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.515765 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.515772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.515772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.515778 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.517941 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.518611 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.518614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.518615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.518681 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.518893 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.518894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.518895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.519078 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.519207 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.519330 (+ 123us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.519392 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":117,"threads_started":1}
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:35.525638 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.518341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:39981 (local address 127.0.0.1:35562)
0807 00:42:35.518393 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.518404 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.518441 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.518621 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.518622 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.518814 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.518883 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.518885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.519680 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.519682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.520496 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.520498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.520933 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.520934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.521018 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.521457 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.521461 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.522730 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.524475 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.524476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.524477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.524694 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.524981 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.524982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.524982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.525035 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.525375 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.525376 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.525423 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.525495 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.525575 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 00:42:35.525767 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.518438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.1.116.189:39981)
0807 00:42:35.518487 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.518489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.518499 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:42:35.518525 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.518540 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.518551 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.518617 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.518980 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.519613 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.520579 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.520844 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.520904 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.522824 (+ 1920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.522826 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.522827 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.522836 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.524382 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.524811 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.524813 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.524813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.524885 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.525117 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.525118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.525118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.525264 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.525378 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.525559 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.525659 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:42:35.526243 20896 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:42:35.530097 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.520482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58990 (local address 127.1.116.190:46181)
0807 00:42:35.520516 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.520518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.520525 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:42:35.520546 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.520547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.520557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.520623 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.521181 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.521727 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.522464 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.522766 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.522856 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.525948 (+ 3092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.525949 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.525950 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.525958 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.528691 (+ 2733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.529121 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.529122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.529123 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.529177 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.529463 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.529464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.529464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.529605 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.529822 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.529870 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.529957 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:42:35.534025 20900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.526517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45718)
0807 00:42:35.526661 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.526674 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.526723 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.526927 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.526928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.527117 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.527165 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.527167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.527971 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.527972 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.528461 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.528462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.528841 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.528842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.528893 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.529350 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.529357 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.530937 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.532924 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.532926 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.532927 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.533194 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.533430 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.533431 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.533431 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.533488 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.533776 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.533776 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.533812 (+ 36us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.533888 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.533965 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260807 00:42:35.534150 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.526653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45718 (local address 127.1.116.188:34939)
0807 00:42:35.526715 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.526717 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.526725 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:42:35.526804 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.526804 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.526814 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.526877 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.527257 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.527884 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.528554 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.528771 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.528844 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.531044 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.531046 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.531046 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.531054 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.532803 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.533294 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.533295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.533296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.533358 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.533553 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.533554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.533554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.533688 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.533742 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.533936 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.534042 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:42:35.536144 20896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952: Bootstrap starting.
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:42:35.537154 20896 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:35.537499 20896 log.cc:824] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:35.538498 20896 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952: No bootstrap required, opened a new log
I20260807 00:42:35.540005 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.531259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35576 (local address 127.1.116.189:39981)
0807 00:42:35.531300 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.531302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.531313 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:42:35.531347 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.531348 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.531358 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.531429 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.532017 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.532750 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.533862 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.534115 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.534191 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.536964 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.536966 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.536966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.536973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.538677 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.539061 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.539062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.539063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.539214 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.539460 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.539461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.539461 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.539600 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.539789 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.539831 (+ 42us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.539909 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:42:35.540719 20896 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.540967 20896 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:35.541046 20896 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4acdef11228f4deeba9cfbe9d2827952, State: Initialized, Role: FOLLOWER
I20260807 00:42:35.541178 20896 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.541445 20905 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.541507 20896 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:35.541530 20905 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:42:35.542332 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 00:42:35.543699 20921 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:42:35.543848 20921 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:42:35.543955 20795 master_runner.cc:186] Error getting master registration for 127.1.116.190:46181: OK, instance_id { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" instance_seqno: 1786063355151438 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:42:35.546909 20795 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:42:35.553128 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.544039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35578 (local address 127.1.116.189:39981)
0807 00:42:35.544086 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.544088 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.544265 (+ 177us) server_negotiation.cc:407] Connection header received
0807 00:42:35.544363 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.544364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.544376 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.544407 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.545227 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.545937 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.547118 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.547335 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.547412 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.549289 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.549291 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.549292 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.549300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.551621 (+ 2321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.552168 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.552169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.552170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.552225 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.552437 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.552438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.552439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.552578 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.552679 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.552916 (+ 237us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.553023 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 00:42:35.553303 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.544286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45734 (local address 127.1.116.188:34939)
0807 00:42:35.544325 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.544326 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.544898 (+ 572us) server_negotiation.cc:407] Connection header received
0807 00:42:35.545005 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.545006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.545017 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.545045 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.545901 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.546628 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.547716 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.548011 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.548131 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.549883 (+ 1752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.549884 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.549884 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.549893 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.551965 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.552385 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.552387 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.552387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.552440 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.552642 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.552643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.552644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.552818 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.552889 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.553094 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.553177 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:42:35.553874 20787 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } attempt: 3
I20260807 00:42:35.557446 20686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.544625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58992 (local address 127.1.116.190:46181)
0807 00:42:35.544666 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.544669 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.547803 (+ 3134us) server_negotiation.cc:407] Connection header received
0807 00:42:35.547895 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.547896 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.547907 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.547939 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.548707 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.549409 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.550422 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.550799 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.550884 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.552717 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.552718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.552719 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.552726 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.555557 (+ 2831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.556195 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.556197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.556198 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.556352 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.556667 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.556668 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.556669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.556848 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.556935 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.557235 (+ 300us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.557344 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:42:35.561555 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.554293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45736 (local address 127.1.116.188:34939)
0807 00:42:35.554367 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.554369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.554376 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:42:35.554412 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.554413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.554425 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.554482 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.554882 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.555328 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.556081 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.556286 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.556359 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.558518 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.558519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.558520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.558566 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.560278 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.560740 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.560741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.560741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.560798 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.561050 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.561051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.561051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.561169 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.561239 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.561435 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.561495 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:42:35.561633 20789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.554117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45736)
0807 00:42:35.554162 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.554171 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.554206 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.554530 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.554531 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.554745 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.554794 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.554796 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.555413 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.555414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.555995 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.555997 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.556371 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.556372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.556429 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.556863 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.556868 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.558408 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.560385 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.560386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.560386 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.560631 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.560891 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.560892 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.560892 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.560964 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.561262 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.561263 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.561314 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.561421 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.561532 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 00:42:35.563757 20787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: Bootstrap starting.
I20260807 00:42:35.564468 20787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:35.564774 20787 log.cc:824] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:35.565786 20787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: No bootstrap required, opened a new log
I20260807 00:42:35.567791 20787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.567930 20787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:35.567967 20787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 095e644e1b7946f68bd5e752576c5788, State: Initialized, Role: FOLLOWER
I20260807 00:42:35.568118 20787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.568457 20787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:35.568424 20933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.568547 20933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [sys.catalog]: This master's current role is: FOLLOWER
W20260807 00:42:35.570602 20948 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:42:35.570665 20948 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:42:35.570735 20691 master_runner.cc:186] Error getting master registration for 127.1.116.190:46181: OK, instance_id { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" instance_seqno: 1786063355151438 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:42:35.573333 20691 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:42:35.598850 20683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } attempt: 3
Aug 07 00:42:35 dist-test-slave-h6n0 krb5kdc[20571](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063355, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:42:35.606735 20902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.599571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45752 (local address 127.1.116.188:34939)
0807 00:42:35.599670 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:35.599672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:35.599680 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:42:35.599703 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:35.599704 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:35.599716 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:35.599763 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:35.600322 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.600867 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.601694 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.601885 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.601963 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.604073 (+ 2110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:35.604075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:35.604076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:35.604083 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:35.605506 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.605877 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.605878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.605879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.605936 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:35.606151 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:35.606152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:35.606152 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:35.606274 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:35.606350 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:35.606517 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 00:42:35.606619 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 00:42:35.606735 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:35.599371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:34939 (local address 127.0.0.1:45752)
0807 00:42:35.599531 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:35.599545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:35.599592 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:35.599897 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:35.599898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:35.600134 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:35.600229 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.600232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.600987 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.600988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.601588 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:35.601590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:35.601970 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:35.601972 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:35.602039 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:35.602606 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:35.602612 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:35.603981 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:35.605610 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.605611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.605612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.605788 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.606007 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:35.606008 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:35.606008 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:35.606075 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:35.606352 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:35.606354 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:35.606402 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 00:42:35.606489 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:42:35.606620 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260807 00:42:35.608834 20683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: Bootstrap starting.
I20260807 00:42:35.609690 20683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:35.610064 20683 log.cc:824] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:35.611124 20683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: No bootstrap required, opened a new log
I20260807 00:42:35.612776 20683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.612926 20683 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:35.612973 20683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4dbb3c140c054bb596e98d083665c2fd, State: Initialized, Role: FOLLOWER
I20260807 00:42:35.613137 20683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.613476 20951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.613561 20683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:35.613628 20951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [sys.catalog]: This master's current role is: FOLLOWER
W20260807 00:42:35.615667 20962 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:42:35.615777 20962 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:42:35.618127 20590 master_runner.cc:263] No leader master found from master 4dbb3c140c054bb596e98d083665c2fd
I20260807 00:42:35.618193 20590 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260807 00:42:35.648537 20617 tablet.cc:2369] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:42:35.748138 20827 tablet.cc:2369] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:42:35.779326 20905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:35.779589 20905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.780027 20905 leader_election.cc:288] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4dbb3c140c054bb596e98d083665c2fd (127.1.116.190:46181), 095e644e1b7946f68bd5e752576c5788 (127.1.116.189:39981)
I20260807 00:42:35.780575 20755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4acdef11228f4deeba9cfbe9d2827952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095e644e1b7946f68bd5e752576c5788" is_pre_election: true
I20260807 00:42:35.780606 20645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4acdef11228f4deeba9cfbe9d2827952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb3c140c054bb596e98d083665c2fd" is_pre_election: true
I20260807 00:42:35.780797 20645 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acdef11228f4deeba9cfbe9d2827952 in term 0.
I20260807 00:42:35.780797 20755 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4acdef11228f4deeba9cfbe9d2827952 in term 0.
I20260807 00:42:35.781016 20825 leader_election.cc:302] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 095e644e1b7946f68bd5e752576c5788, 4acdef11228f4deeba9cfbe9d2827952; no voters:
I20260807 00:42:35.781196 20905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:42:35.781260 20905 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:42:35.781286 20905 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:35.782898 20905 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.783100 20905 leader_election.cc:288] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [CANDIDATE]: Term 1 election: Requested vote from peers 4dbb3c140c054bb596e98d083665c2fd (127.1.116.190:46181), 095e644e1b7946f68bd5e752576c5788 (127.1.116.189:39981)
I20260807 00:42:35.783324 20755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4acdef11228f4deeba9cfbe9d2827952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "095e644e1b7946f68bd5e752576c5788"
I20260807 00:42:35.783388 20645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4acdef11228f4deeba9cfbe9d2827952" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4dbb3c140c054bb596e98d083665c2fd"
I20260807 00:42:35.783465 20755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:35.783480 20645 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:35.784572 20755 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acdef11228f4deeba9cfbe9d2827952 in term 1.
I20260807 00:42:35.784634 20645 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4acdef11228f4deeba9cfbe9d2827952 in term 1.
I20260807 00:42:35.784921 20820 leader_election.cc:302] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4acdef11228f4deeba9cfbe9d2827952, 4dbb3c140c054bb596e98d083665c2fd; no voters:
I20260807 00:42:35.785041 20905 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:35.785362 20905 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [term 1 LEADER]: Becoming Leader. State: Replica: 4acdef11228f4deeba9cfbe9d2827952, State: Running, Role: LEADER
I20260807 00:42:35.785514 20905 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } }
I20260807 00:42:35.786063 20966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acdef11228f4deeba9cfbe9d2827952. Latest consensus state: current_term: 1 leader_uuid: "4acdef11228f4deeba9cfbe9d2827952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.786149 20966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:35.786407 20968 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:35.786670 20968 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:35.788471 20645 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [term 1 FOLLOWER]: Refusing update from remote peer 4acdef11228f4deeba9cfbe9d2827952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:35.788479 20755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [term 1 FOLLOWER]: Refusing update from remote peer 4acdef11228f4deeba9cfbe9d2827952: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:35.788908 20905 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:35.789049 20966 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [LEADER]: Connected to new peer: Peer: permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:35.790388 20933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acdef11228f4deeba9cfbe9d2827952. Latest consensus state: current_term: 1 leader_uuid: "4acdef11228f4deeba9cfbe9d2827952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.790488 20933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:42:35.790593 20951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4acdef11228f4deeba9cfbe9d2827952. Latest consensus state: current_term: 1 leader_uuid: "4acdef11228f4deeba9cfbe9d2827952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.790712 20951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:42:35.791643 20966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4acdef11228f4deeba9cfbe9d2827952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.791738 20966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:35.791989 20933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4acdef11228f4deeba9cfbe9d2827952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.792088 20933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:42:35.792060 20966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4acdef11228f4deeba9cfbe9d2827952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.792127 20966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:35.792551 20968 catalog_manager.cc:1347] Generated new cluster ID: 4002fed8289f442f94560f0d63b357ac
I20260807 00:42:35.792586 20970 mvcc.cc:205] Tried to move back new op lower bound from 7315715505306161152 to 7315715505298202624. Current Snapshot: MvccSnapshot[applied={T|T < 7315715505306161152}]
I20260807 00:42:35.792649 20968 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:42:35.792685 20951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4acdef11228f4deeba9cfbe9d2827952" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4dbb3c140c054bb596e98d083665c2fd" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 46181 } } peers { permanent_uuid: "095e644e1b7946f68bd5e752576c5788" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 39981 } } peers { permanent_uuid: "4acdef11228f4deeba9cfbe9d2827952" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 34939 } } }
I20260807 00:42:35.792753 20951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:42:35.823963 20968 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:42:35.824724 20968 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:42:35.834789 20968 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952: Generated new TSK 0
I20260807 00:42:35.835037 20968 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:42:36.571201 20948 catalog_manager.cc:1259] Loaded cluster ID: 4002fed8289f442f94560f0d63b357ac
I20260807 00:42:36.571327 20948 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: loading cluster ID for follower catalog manager: success
I20260807 00:42:36.572292 20948 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: acquiring CA information for follower catalog manager: success
I20260807 00:42:36.572611 20948 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 095e644e1b7946f68bd5e752576c5788: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:42:36.616305 20962 catalog_manager.cc:1259] Loaded cluster ID: 4002fed8289f442f94560f0d63b357ac
I20260807 00:42:36.616410 20962 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: loading cluster ID for follower catalog manager: success
I20260807 00:42:36.617234 20962 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: acquiring CA information for follower catalog manager: success
I20260807 00:42:36.617555 20962 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4dbb3c140c054bb596e98d083665c2fd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:42:36.893000 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 20590
I20260807 00:42:36.908495 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 20691
I20260807 00:42:36.915663 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:36.914868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:46181 (local address 127.0.0.1:59004)
0807 00:42:36.915483 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:36.915568 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:46181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":503,"threads_started":1}
W20260807 00:42:36.915871 20820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.190:46181: connect: Connection refused (error 111)
I20260807 00:42:36.916368 20989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:36.916257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:46181 (local address 127.0.0.1:59020)
0807 00:42:36.916310 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:36.916338 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:46181: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 00:42:36.916471 20820 consensus_peers.cc:487] T 00000000000000000000000000000000 P 4acdef11228f4deeba9cfbe9d2827952 -> Peer 4dbb3c140c054bb596e98d083665c2fd (127.1.116.190:46181): Couldn't send request to peer 4dbb3c140c054bb596e98d083665c2fd. Status: Network error: Client connection negotiation failed: client connection to 127.1.116.190:46181: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 00:42:36.921777 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 20795
2026-08-07T00:42:36Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3278 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:42:36 dist-test-slave-h6n0 krb5kdc[20994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:42:36 dist-test-slave-h6n0 krb5kdc[20994](info): set up 2 sockets
Aug 07 00:42:36 dist-test-slave-h6n0 krb5kdc[20994](info): commencing operation
krb5kdc: starting...
W20260807 00:42:38.172058 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:42:38 dist-test-slave-h6n0 krb5kdc[20994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:42:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:42:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:42:38.314194 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43975
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:43975
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:38.404325 21016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:38.404556 21016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:38.404583 21016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:38.405678 21016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:38.405715 21016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:38.405730 21016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:38.405742 21016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:38.405754 21016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:38.407279 21016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:43975
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:38.407490 21016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:38.407691 21016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:38.413774 21024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:38.413786 21026 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:38.413789 21023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:38.414059 21016 server_base.cc:1034] running on GCE node
I20260807 00:42:38.414806 21016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:38.415140 21016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:38.416327 21016 hybrid_clock.cc:648] HybridClock initialized: now 1786063358416307 us; error 37 us; skew 500 ppm
Aug 07 00:42:38 dist-test-slave-h6n0 krb5kdc[20994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063358, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:38.419281 21016 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:38.419776 21016 webserver.cc:460] Webserver started at http://127.1.116.190:38349/ using document root <none> and password file <none>
I20260807 00:42:38.420017 21016 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:38.420090 21016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:38.420250 21016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:38.421312 21016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "024dc6fa795b443eb21d9459de806e14"
format_stamp: "Formatted at 2026-08-07 00:42:38 on dist-test-slave-h6n0"
server_key: "9435c380156fe2ec37ca52718c64c156"
server_key_iv: "934193181ea01371592b9f74641555da"
server_key_version: "encryptionkey@0"
I20260807 00:42:38.421695 21016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "024dc6fa795b443eb21d9459de806e14"
format_stamp: "Formatted at 2026-08-07 00:42:38 on dist-test-slave-h6n0"
server_key: "9435c380156fe2ec37ca52718c64c156"
server_key_iv: "934193181ea01371592b9f74641555da"
server_key_version: "encryptionkey@0"
I20260807 00:42:38.423251 21016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:42:38.424233 21033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:38.424521 21016 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:38.424659 21016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "024dc6fa795b443eb21d9459de806e14"
format_stamp: "Formatted at 2026-08-07 00:42:38 on dist-test-slave-h6n0"
server_key: "9435c380156fe2ec37ca52718c64c156"
server_key_iv: "934193181ea01371592b9f74641555da"
server_key_version: "encryptionkey@0"
I20260807 00:42:38.424736 21016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:38.439087 21016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:38.441956 21016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:38.442284 21016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:38.447340 21016 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:43975
I20260807 00:42:38.447407 21114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:43975 every 8 connection(s)
I20260807 00:42:38.447788 21016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:38.448904 21115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:38.451779 21115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14: Bootstrap starting.
I20260807 00:42:38.451892 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 21016
I20260807 00:42:38.452029 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:42:38.452261 1490 external_mini_cluster.cc:1428] Setting key be1fe9aa3f45c8c61de0785ba64eeb7c
I20260807 00:42:38.452692 21115 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:38.453132 21115 log.cc:824] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:38.454082 21115 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14: No bootstrap required, opened a new log
I20260807 00:42:38.456773 21115 raft_consensus.cc:348] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024dc6fa795b443eb21d9459de806e14" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43975 } }
I20260807 00:42:38.456928 21115 raft_consensus.cc:374] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:38.456952 21115 raft_consensus.cc:729] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 024dc6fa795b443eb21d9459de806e14, State: Initialized, Role: FOLLOWER
I20260807 00:42:38.457110 21115 consensus_queue.cc:260] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024dc6fa795b443eb21d9459de806e14" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43975 } }
I20260807 00:42:38.457204 21115 raft_consensus.cc:388] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:42:38.457230 21115 raft_consensus.cc:482] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:42:38.457309 21115 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:38.458148 21115 raft_consensus.cc:504] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024dc6fa795b443eb21d9459de806e14" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43975 } }
I20260807 00:42:38.458280 21115 leader_election.cc:302] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 024dc6fa795b443eb21d9459de806e14; no voters:
I20260807 00:42:38.458477 21115 leader_election.cc:288] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:42:38.458509 21123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:38.458663 21123 raft_consensus.cc:686] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [term 1 LEADER]: Becoming Leader. State: Replica: 024dc6fa795b443eb21d9459de806e14, State: Running, Role: LEADER
I20260807 00:42:38.458811 21115 sys_catalog.cc:564] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:38.458803 21123 consensus_queue.cc:237] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024dc6fa795b443eb21d9459de806e14" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43975 } }
I20260807 00:42:38.460657 21125 sys_catalog.cc:455] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 024dc6fa795b443eb21d9459de806e14. Latest consensus state: current_term: 1 leader_uuid: "024dc6fa795b443eb21d9459de806e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024dc6fa795b443eb21d9459de806e14" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43975 } } }
I20260807 00:42:38.460852 21124 sys_catalog.cc:455] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "024dc6fa795b443eb21d9459de806e14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "024dc6fa795b443eb21d9459de806e14" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43975 } } }
I20260807 00:42:38.460896 21125 sys_catalog.cc:458] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:38.461014 21124 sys_catalog.cc:458] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14 [sys.catalog]: This master's current role is: LEADER
Aug 07 00:42:38 dist-test-slave-h6n0 krb5kdc[20994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
W20260807 00:42:38.461287 21143 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:42:38.461411 21143 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:42:38.461505 21149 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:38.461755 21149 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:38.464263 21149 catalog_manager.cc:1347] Generated new cluster ID: d98ebb9fc4cc4c64b9990c4b1054813a
I20260807 00:42:38.464334 21149 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:42:38.466048 21119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:38.452923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45558 (local address 127.1.116.190:43975)
0807 00:42:38.453182 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:38.453194 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:38.453242 (+ 48us) server_negotiation.cc:407] Connection header received
0807 00:42:38.453938 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:38.453950 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:38.454019 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:38.454216 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:38.454972 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:38.456528 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:38.457551 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:38.458209 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:38.458336 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:38.461800 (+ 3464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:38.461805 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:38.461805 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:38.461820 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:38.464088 (+ 2268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:38.464618 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:38.464622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:38.464622 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:38.464733 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:38.464963 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:38.464964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:38.464964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:38.465251 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:38.465466 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:38.465551 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 00:42:38.465830 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":96,"threads_started":1}
I20260807 00:42:38.471814 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:42:38.480080 21149 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:42:38.480873 21149 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:42:38.490721 21149 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 024dc6fa795b443eb21d9459de806e14: Generated new TSK 0
I20260807 00:42:38.490944 21149 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:42:38.526518 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 21016
2026-08-07T00:42:38Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1609 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:42:38 dist-test-slave-h6n0 krb5kdc[21156](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:42:38 dist-test-slave-h6n0 krb5kdc[21156](info): set up 2 sockets
Aug 07 00:42:38 dist-test-slave-h6n0 krb5kdc[21156](info): commencing operation
krb5kdc: starting...
W20260807 00:42:39.802949 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:42:39 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:42:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:42:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:42:39.941679 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36905
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:40.024699 21173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:40.024933 21173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:40.025244 21173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:40.026481 21173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:40.026573 21173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:40.026602 21173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:40.026630 21173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:40.026661 21173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:40.028220 21173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:40.028501 21173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:40.028736 21173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:40.033948 21186 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:40.033977 21180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:40.034220 21173 server_base.cc:1034] running on GCE node
W20260807 00:42:40.033948 21182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:40.034612 21173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:40.034962 21173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:40.036144 21173 hybrid_clock.cc:648] HybridClock initialized: now 1786063360036135 us; error 47 us; skew 500 ppm
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:40.038859 21173 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:40.039356 21173 webserver.cc:460] Webserver started at http://127.1.116.190:41219/ using document root <none> and password file <none>
I20260807 00:42:40.039616 21173 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:40.039687 21173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:40.039839 21173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:40.040872 21173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "830586726b8d4ecebdf880f66936be53"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "c4b6e95c4c4fb1987e33f31c55066eeb"
server_key_iv: "e04541638df2b20a64989558be8eb1ab"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.041229 21173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "830586726b8d4ecebdf880f66936be53"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "c4b6e95c4c4fb1987e33f31c55066eeb"
server_key_iv: "e04541638df2b20a64989558be8eb1ab"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.042654 21173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:42:40.043480 21195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.043710 21173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:40.043823 21173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "830586726b8d4ecebdf880f66936be53"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "c4b6e95c4c4fb1987e33f31c55066eeb"
server_key_iv: "e04541638df2b20a64989558be8eb1ab"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.043906 21173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:40.061789 21173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:40.063977 21173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:40.064193 21173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:40.069597 21173 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:36905
I20260807 00:42:40.069681 21274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:36905 every 8 connection(s)
I20260807 00:42:40.070036 21173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:40.071321 21275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:40.074002 21275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Bootstrap starting.
I20260807 00:42:40.074707 21275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:40.075013 21275 log.cc:824] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:40.075831 21275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: No bootstrap required, opened a new log
I20260807 00:42:40.077600 21275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:40.077759 21275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:40.077783 21275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830586726b8d4ecebdf880f66936be53, State: Initialized, Role: FOLLOWER
I20260807 00:42:40.077903 21275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:40.077991 21275 raft_consensus.cc:388] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:42:40.078024 21275 raft_consensus.cc:482] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:42:40.078090 21275 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:40.078383 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 21173
I20260807 00:42:40.078509 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:42:40.078792 1490 external_mini_cluster.cc:1428] Setting key ee9cc37666659bb25419d9367f2c44c1
I20260807 00:42:40.079455 21275 raft_consensus.cc:504] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:40.079692 21275 leader_election.cc:302] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 830586726b8d4ecebdf880f66936be53; no voters:
I20260807 00:42:40.079895 21275 leader_election.cc:288] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:42:40.079960 21280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:40.080168 21280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 LEADER]: Becoming Leader. State: Replica: 830586726b8d4ecebdf880f66936be53, State: Running, Role: LEADER
I20260807 00:42:40.080227 21275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:40.080339 21280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:40.080933 21281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "830586726b8d4ecebdf880f66936be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } } }
I20260807 00:42:40.080968 21282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 830586726b8d4ecebdf880f66936be53. Latest consensus state: current_term: 1 leader_uuid: "830586726b8d4ecebdf880f66936be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } } }
I20260807 00:42:40.081033 21281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:40.081033 21282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:40.081420 21291 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:40.082149 21291 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:40.084972 21291 catalog_manager.cc:1347] Generated new cluster ID: 7a0071b36599428498c6e11fed1df519
I20260807 00:42:40.085047 21291 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:40.090442 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.079409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.1.116.190:36905)
0807 00:42:40.079616 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.079623 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.079651 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:42:40.079775 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.079781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.079851 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.080080 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:40.080870 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.082394 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.083459 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.084048 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.084176 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.086960 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:40.086965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:40.086965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:40.086985 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:40.088820 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.089222 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.089226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.089227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.089339 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.089564 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.089565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.089566 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.089835 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:40.090027 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.090098 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.090258 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":130,"threads_started":1}
I20260807 00:42:40.096890 21291 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:42:40.097599 21291 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:42:40.104616 21291 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Generated new TSK 0
I20260807 00:42:40.104879 21291 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:42:40.143399 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36905
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:40.259835 21312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:40.260085 21312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:40.260113 21312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:40.261404 21312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:40.261454 21312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:40.261482 21312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:42:40.263154 21312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:40.263408 21312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:40.263633 21312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:40.269184 21321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:40.269136 21323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:40.269158 21320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:40.269588 21312 server_base.cc:1034] running on GCE node
I20260807 00:42:40.269814 21312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:40.270113 21312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:40.271303 21312 hybrid_clock.cc:648] HybridClock initialized: now 1786063360271269 us; error 40 us; skew 500 ppm
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:40.284893 21312 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:42:40.285483 21312 webserver.cc:460] Webserver started at http://127.1.116.129:40951/ using document root <none> and password file <none>
I20260807 00:42:40.285790 21312 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:40.285868 21312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:40.286029 21312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:40.287480 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "cccd6eece80e43b89de54997299a5a12"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "fb682f74a1b5763feb61665536419fd1"
server_key_iv: "18d0051c7b7c8750b70a19bb4f46c1bb"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.287914 21312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "cccd6eece80e43b89de54997299a5a12"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "fb682f74a1b5763feb61665536419fd1"
server_key_iv: "18d0051c7b7c8750b70a19bb4f46c1bb"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.289883 21312 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260807 00:42:40.290675 21336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.290907 21312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:40.291024 21312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "cccd6eece80e43b89de54997299a5a12"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "fb682f74a1b5763feb61665536419fd1"
server_key_iv: "18d0051c7b7c8750b70a19bb4f46c1bb"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.291107 21312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:40.301363 21312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:40.303606 21312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:40.303768 21312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:40.304004 21312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:40.304318 21312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:40.304381 21312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.304418 21312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:40.304445 21312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.311190 21312 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:33775
I20260807 00:42:40.311254 21486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:33775 every 8 connection(s)
I20260807 00:42:40.311537 21312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:42:40.313215 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 21312
I20260807 00:42:40.313386 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:42:40.313663 1490 external_mini_cluster.cc:1428] Setting key d142055e8b9f5c15c14b4c7f1c6bb5fb
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:40.324378 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.312374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:44543 (local address 127.1.116.190:36905)
0807 00:42:40.312460 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.312463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.312711 (+ 248us) server_negotiation.cc:407] Connection header received
0807 00:42:40.313262 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.313263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.313281 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.313330 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:40.314720 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.315308 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.317047 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.317358 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.317443 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.320284 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:40.320286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:40.320287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:40.320296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:40.322273 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.322892 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.322894 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.322894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.322984 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.323365 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.323367 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.323368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.323604 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:40.323686 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.324161 (+ 475us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.324266 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:42:40.324595 21490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.312364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:44543)
0807 00:42:40.312584 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.312620 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.313133 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.313420 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.313423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.314310 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:40.314606 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.314612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.315443 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.315446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.316934 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.316938 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.317461 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.317462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.317577 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.318248 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:40.318258 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:40.320158 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:40.322378 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:40.322388 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:40.322391 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:40.322781 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:40.323070 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:40.323088 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:40.323089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:40.323243 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:40.323731 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:40.323739 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:40.323944 (+ 205us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.324070 (+ 126us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.324288 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":120,"threads_started":1}
I20260807 00:42:40.325208 21487 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36905
I20260807 00:42:40.325328 21487 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:42:40.325554 21487 heartbeater.cc:511] Master 127.1.116.190:36905 requested a full tablet report, sending...
I20260807 00:42:40.326148 21225 ts_manager.cc:194] Registered new tserver with Master: cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775)
I20260807 00:42:40.326850 21225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:44543
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:42:40.343541 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36905
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:40.434798 21495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:40.435048 21495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:40.435087 21495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:40.436208 21495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:40.436280 21495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:40.436326 21495 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:42:40.437808 21495 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:40.438076 21495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:40.438371 21495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:40.443661 21504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:40.443661 21507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:40.443900 21495 server_base.cc:1034] running on GCE node
W20260807 00:42:40.443672 21503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:40.444188 21495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:40.444526 21495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:40.445730 21495 hybrid_clock.cc:648] HybridClock initialized: now 1786063360445692 us; error 56 us; skew 500 ppm
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:40.448580 21495 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:42:40.449090 21495 webserver.cc:460] Webserver started at http://127.1.116.130:34865/ using document root <none> and password file <none>
I20260807 00:42:40.449344 21495 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:40.449415 21495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:40.449571 21495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:40.450683 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "726e3fcb13a840b39434fd349f5f0558"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "03e5af14091c2fca59d9fbbd81532dd4"
server_key_iv: "076353fa69c5917689be2d016c0f0d9f"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.451047 21495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "726e3fcb13a840b39434fd349f5f0558"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "03e5af14091c2fca59d9fbbd81532dd4"
server_key_iv: "076353fa69c5917689be2d016c0f0d9f"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.452515 21495 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:42:40.453397 21515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.453679 21495 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:40.453785 21495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "726e3fcb13a840b39434fd349f5f0558"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "03e5af14091c2fca59d9fbbd81532dd4"
server_key_iv: "076353fa69c5917689be2d016c0f0d9f"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.453871 21495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:40.463466 21495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:40.465526 21495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:40.465699 21495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:40.465939 21495 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:40.466269 21495 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:40.466327 21495 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.466370 21495 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:40.466400 21495 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.473749 21495 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:41959
I20260807 00:42:40.473799 21673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:41959 every 8 connection(s)
I20260807 00:42:40.474138 21495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:42:40.480301 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 21495
I20260807 00:42:40.480456 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:42:40.480707 1490 external_mini_cluster.cc:1428] Setting key 29cf853e233605e073f3d197ab7907fe
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:40.485600 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.475102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:43363 (local address 127.1.116.190:36905)
0807 00:42:40.475204 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.475208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.475452 (+ 244us) server_negotiation.cc:407] Connection header received
0807 00:42:40.476118 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.476120 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.476135 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.476190 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:40.477315 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.477874 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.479069 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.479280 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.479363 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.482392 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:40.482393 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:40.482394 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:40.482402 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:40.483929 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.484373 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.484374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.484375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.484436 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.484704 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.484705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.484705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.484879 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:40.484964 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.485328 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.485448 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:42:40.485631 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.474990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:43363)
0807 00:42:40.475236 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.475322 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.475984 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.476279 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.476284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.476967 (+ 683us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:40.477204 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.477209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.477983 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.477984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.478958 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.478961 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.479372 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.479373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.479515 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.480197 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:40.480209 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:40.482288 (+ 2079us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:40.484027 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:40.484035 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:40.484037 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:40.484288 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:40.484518 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:40.484519 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:40.484519 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:40.484633 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:40.484988 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:40.484995 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:40.485165 (+ 170us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.485278 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.485455 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":100,"threads_started":1}
I20260807 00:42:40.486291 21674 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36905
I20260807 00:42:40.486395 21674 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:40.486629 21674 heartbeater.cc:511] Master 127.1.116.190:36905 requested a full tablet report, sending...
I20260807 00:42:40.487082 21225 ts_manager.cc:194] Registered new tserver with Master: 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959)
I20260807 00:42:40.487519 21225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:43363
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:42:40.509585 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36905
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:40.602865 21682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:40.603132 21682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:40.603171 21682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:40.604406 21682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:40.604473 21682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:40.604548 21682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:42:40.606269 21682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:40.606638 21682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:40.606904 21682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:40.612931 21690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:40.612939 21688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:40.612939 21692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:40.613377 21682 server_base.cc:1034] running on GCE node
I20260807 00:42:40.613636 21682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:40.613911 21682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:40.615083 21682 hybrid_clock.cc:648] HybridClock initialized: now 1786063360615071 us; error 40 us; skew 500 ppm
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:40.617826 21682 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:42:40.618359 21682 webserver.cc:460] Webserver started at http://127.1.116.131:40953/ using document root <none> and password file <none>
I20260807 00:42:40.618674 21682 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:40.618747 21682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:40.618897 21682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:40.620115 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "60bbc58eee394409ad018c34418ca4c3"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "82352960d86dd291eab09099bd4266a1"
server_key_iv: "447e45b63e0a6ee9d3d65b3b99ba1977"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.620509 21682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "60bbc58eee394409ad018c34418ca4c3"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "82352960d86dd291eab09099bd4266a1"
server_key_iv: "447e45b63e0a6ee9d3d65b3b99ba1977"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.621987 21682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:42:40.622854 21702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.623081 21682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:42:40.623190 21682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "60bbc58eee394409ad018c34418ca4c3"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "82352960d86dd291eab09099bd4266a1"
server_key_iv: "447e45b63e0a6ee9d3d65b3b99ba1977"
server_key_version: "encryptionkey@0"
I20260807 00:42:40.623255 21682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:40.638906 21682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:40.641105 21682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:40.641227 21682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:40.641423 21682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:40.641782 21682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:40.641819 21682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.641841 21682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:40.641896 21682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:40.649363 21682 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:40405
I20260807 00:42:40.649411 21866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:40405 every 8 connection(s)
I20260807 00:42:40.649801 21682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:42:40.656137 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 21682
I20260807 00:42:40.656246 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:42:40.656471 1490 external_mini_cluster.cc:1428] Setting key a81f034af247f8bbc09abab397684c8b
Aug 07 00:42:40 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063360, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:40.660712 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.650639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:46031 (local address 127.1.116.190:36905)
0807 00:42:40.650732 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.650735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.651030 (+ 295us) server_negotiation.cc:407] Connection header received
0807 00:42:40.651573 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.651575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.651588 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.651674 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:40.652711 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.653245 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.654325 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.654609 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.654710 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.657484 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:40.657485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:40.657486 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:40.657494 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:40.659276 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.659712 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.659713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.659713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.659771 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.660077 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.660078 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.660078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.660192 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:40.660248 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.660543 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.660612 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:42:40.660838 21869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.650642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:46031)
0807 00:42:40.650882 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.650926 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.651456 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.651763 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.651767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.652380 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:40.652619 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.652624 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.653335 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.653336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.654173 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.654175 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.654736 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.654737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.654850 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.655384 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:40.655392 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:40.657290 (+ 1898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:40.659378 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:40.659384 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:40.659386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:40.659632 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:40.659869 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:40.659870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:40.659871 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:40.660006 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:40.660280 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:40.660286 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:40.660408 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.660511 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.660635 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":108,"threads_started":1}
I20260807 00:42:40.661303 21867 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36905
I20260807 00:42:40.661440 21867 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:40.661672 21867 heartbeater.cc:511] Master 127.1.116.190:36905 requested a full tablet report, sending...
I20260807 00:42:40.662170 21225 ts_manager.cc:194] Registered new tserver with Master: 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405)
I20260807 00:42:40.662629 21225 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:46031
I20260807 00:42:40.668498 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:42:40.676947 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.669844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37138 (local address 127.1.116.190:36905)
0807 00:42:40.669893 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.669895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.670047 (+ 152us) server_negotiation.cc:407] Connection header received
0807 00:42:40.670104 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.670105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.670114 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.670162 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:40.670941 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.671421 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.672304 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.672560 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.672636 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.673914 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:40.673915 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:40.673915 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:40.673922 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:40.675762 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.676128 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.676128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.676129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.676183 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:40.676392 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:40.676393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:40.676393 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:40.676541 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:40.676622 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.676785 (+ 163us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.676878 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:42:40.678880 21225 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:42:40.680735 21225 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:42:40.689035 21886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.684264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:41959 (local address 127.0.0.1:57208)
0807 00:42:40.684726 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.684751 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.684806 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.685173 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.685178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.685181 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:40.685270 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.685273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.686310 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.686311 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.687675 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.687678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.688711 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.688711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.688792 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.688807 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.688873 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.688935 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":145,"threads_started":1}
I20260807 00:42:40.689256 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.684817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.1.116.129:33775)
0807 00:42:40.685117 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.685122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.685140 (+ 18us) server_negotiation.cc:407] Connection header received
0807 00:42:40.685188 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.685190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.685209 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.685326 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:40.685490 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.686643 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.687983 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.688743 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.688840 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.688893 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.689000 (+ 107us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.689048 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":101,"threads_started":1}
I20260807 00:42:40.689433 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.684520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57208 (local address 127.1.116.130:41959)
0807 00:42:40.684831 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.684838 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.684852 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:42:40.684910 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.684913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.684933 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.685031 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:40.685374 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.686183 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.687783 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.688604 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.688689 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.688760 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.689281 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.689334 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":112,"threads_started":1}
I20260807 00:42:40.689731 21888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.684259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33775 (local address 127.0.0.1:42776)
0807 00:42:40.684455 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.684508 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.684596 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.685343 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.685344 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.685346 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:40.685403 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.685405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.686758 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.686759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.687865 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.687867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.688849 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.688850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.688921 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.688925 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.688984 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.689034 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":118,"threads_started":1}
I20260807 00:42:40.690357 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.684260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:40405 (local address 127.0.0.1:47478)
0807 00:42:40.684727 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.684736 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.684951 (+ 215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.685234 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.685236 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.685240 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:40.685312 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.685315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.687791 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.687791 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.688575 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.688578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.689872 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.689873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.690159 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.690164 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.690245 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.690292 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":2,"thread_start_us":78,"threads_started":1}
I20260807 00:42:40.690408 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.684567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47478 (local address 127.1.116.131:40405)
0807 00:42:40.685058 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.685061 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.685072 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:42:40.685113 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.685115 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.685132 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.685225 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:40.685406 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.686653 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.688675 (+ 2022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.689419 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.689497 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.689554 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.690263 (+ 709us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.690315 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":123,"threads_started":1}
I20260807 00:42:40.690405 21577 tablet_service.cc:1467] Processing CreateTablet for tablet 8108980236184829a450823433bee975 (DEFAULT_TABLE table=test-table [id=2d44a327fd2948aeb3d574a1ac60d290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:40.690482 21396 tablet_service.cc:1467] Processing CreateTablet for tablet 8108980236184829a450823433bee975 (DEFAULT_TABLE table=test-table [id=2d44a327fd2948aeb3d574a1ac60d290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:40.690881 21396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8108980236184829a450823433bee975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:40.690881 21577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8108980236184829a450823433bee975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:40.691388 21770 tablet_service.cc:1467] Processing CreateTablet for tablet 8108980236184829a450823433bee975 (DEFAULT_TABLE table=test-table [id=2d44a327fd2948aeb3d574a1ac60d290]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:40.691715 21770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8108980236184829a450823433bee975. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:40.694993 21897 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Bootstrap starting.
I20260807 00:42:40.695140 21900 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Bootstrap starting.
I20260807 00:42:40.695740 21897 tablet_bootstrap.cc:654] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:40.695802 21900 tablet_bootstrap.cc:654] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:40.696264 21897 log.cc:824] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:40.696338 21900 log.cc:824] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:40.696712 21899 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Bootstrap starting.
I20260807 00:42:40.697060 21897 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: No bootstrap required, opened a new log
I20260807 00:42:40.697167 21897 ts_tablet_manager.cc:1397] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:42:40.697359 21899 tablet_bootstrap.cc:654] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:40.697889 21899 log.cc:824] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:40.698767 21900 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: No bootstrap required, opened a new log
I20260807 00:42:40.698812 21899 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: No bootstrap required, opened a new log
I20260807 00:42:40.698856 21900 ts_tablet_manager.cc:1397] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 00:42:40.698874 21899 ts_tablet_manager.cc:1397] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:42:40.699234 21897 raft_consensus.cc:348] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.699417 21897 raft_consensus.cc:374] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:40.699463 21897 raft_consensus.cc:729] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60bbc58eee394409ad018c34418ca4c3, State: Initialized, Role: FOLLOWER
I20260807 00:42:40.699611 21897 consensus_queue.cc:260] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.699946 21867 heartbeater.cc:503] Master 127.1.116.190:36905 was elected leader, sending a full tablet report...
I20260807 00:42:40.699952 21897 ts_tablet_manager.cc:1428] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:42:40.701320 21900 raft_consensus.cc:348] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.701320 21899 raft_consensus.cc:348] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.701534 21900 raft_consensus.cc:374] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:40.701534 21899 raft_consensus.cc:374] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:40.701577 21900 raft_consensus.cc:729] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 726e3fcb13a840b39434fd349f5f0558, State: Initialized, Role: FOLLOWER
I20260807 00:42:40.701577 21899 raft_consensus.cc:729] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cccd6eece80e43b89de54997299a5a12, State: Initialized, Role: FOLLOWER
I20260807 00:42:40.701718 21899 consensus_queue.cc:260] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.701718 21900 consensus_queue.cc:260] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.702036 21899 ts_tablet_manager.cc:1428] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:42:40.702042 21900 ts_tablet_manager.cc:1428] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:42:40.702065 21487 heartbeater.cc:503] Master 127.1.116.190:36905 was elected leader, sending a full tablet report...
I20260807 00:42:40.702178 21674 heartbeater.cc:503] Master 127.1.116.190:36905 was elected leader, sending a full tablet report...
W20260807 00:42:40.724805 21676 tablet.cc:2369] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:42:40.812450 21488 tablet.cc:2369] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:42:40.900565 21868 tablet.cc:2369] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:42:40.937693 21908 raft_consensus.cc:482] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:40.937855 21908 raft_consensus.cc:504] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.938251 21908 leader_election.cc:288] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775), 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405)
I20260807 00:42:40.941824 21894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.938711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:48499 (local address 127.1.116.131:40405)
0807 00:42:40.938769 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.938772 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.939046 (+ 274us) server_negotiation.cc:407] Connection header received
0807 00:42:40.939099 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.939100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.939112 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.939216 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:40.939406 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.939941 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.941017 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.941426 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.941525 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.941538 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.941706 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.941746 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 00:42:40.941855 21911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.938643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:40405 (local address 127.1.116.130:48499)
0807 00:42:40.938915 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.938932 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.939023 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.939220 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.939221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.939222 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:40.939297 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.939299 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.940042 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.940043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.940910 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.940913 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.941540 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.941542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.941620 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.941624 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.941699 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.941748 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":21,"thread_start_us":147,"threads_started":1}
I20260807 00:42:40.941972 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.938696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:40691 (local address 127.1.116.129:33775)
0807 00:42:40.938772 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.938774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.938791 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:42:40.938904 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.938905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.938918 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.939032 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:40.939498 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.940094 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.941120 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.941538 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.941650 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.941662 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.941839 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.941885 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:42:40.941972 21677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.938643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33775 (local address 127.1.116.130:40691)
0807 00:42:40.938723 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:40.938739 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:40.938801 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:40.939287 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:40.939289 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:40.939291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:40.939374 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.939376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.940198 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.940199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.941011 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:40.941013 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.941649 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.941651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:40.941743 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.941747 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:42:40.941817 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 00:42:40.941873 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 00:42:40.942580 21425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8108980236184829a450823433bee975" candidate_uuid: "726e3fcb13a840b39434fd349f5f0558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cccd6eece80e43b89de54997299a5a12" is_pre_election: true
I20260807 00:42:40.942588 21797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8108980236184829a450823433bee975" candidate_uuid: "726e3fcb13a840b39434fd349f5f0558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60bbc58eee394409ad018c34418ca4c3" is_pre_election: true
I20260807 00:42:40.942777 21425 raft_consensus.cc:2454] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 726e3fcb13a840b39434fd349f5f0558 in term 0.
I20260807 00:42:40.942817 21797 raft_consensus.cc:2454] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 726e3fcb13a840b39434fd349f5f0558 in term 0.
I20260807 00:42:40.943156 21519 leader_election.cc:302] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60bbc58eee394409ad018c34418ca4c3, 726e3fcb13a840b39434fd349f5f0558; no voters:
I20260807 00:42:40.943317 21908 raft_consensus.cc:2781] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:42:40.943380 21908 raft_consensus.cc:482] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:42:40.943410 21908 raft_consensus.cc:3037] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:40.944205 21908 raft_consensus.cc:504] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.944373 21908 leader_election.cc:288] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [CANDIDATE]: Term 1 election: Requested vote from peers cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775), 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405)
I20260807 00:42:40.944576 21425 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8108980236184829a450823433bee975" candidate_uuid: "726e3fcb13a840b39434fd349f5f0558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cccd6eece80e43b89de54997299a5a12"
I20260807 00:42:40.944608 21797 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8108980236184829a450823433bee975" candidate_uuid: "726e3fcb13a840b39434fd349f5f0558" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60bbc58eee394409ad018c34418ca4c3"
I20260807 00:42:40.944661 21425 raft_consensus.cc:3037] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:40.944698 21797 raft_consensus.cc:3037] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:40.945591 21797 raft_consensus.cc:2454] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 726e3fcb13a840b39434fd349f5f0558 in term 1.
I20260807 00:42:40.945585 21425 raft_consensus.cc:2454] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 726e3fcb13a840b39434fd349f5f0558 in term 1.
I20260807 00:42:40.945781 21520 leader_election.cc:302] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 726e3fcb13a840b39434fd349f5f0558, cccd6eece80e43b89de54997299a5a12; no voters:
I20260807 00:42:40.945892 21908 raft_consensus.cc:2781] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:40.946065 21908 raft_consensus.cc:686] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 1 LEADER]: Becoming Leader. State: Replica: 726e3fcb13a840b39434fd349f5f0558, State: Running, Role: LEADER
I20260807 00:42:40.946169 21908 consensus_queue.cc:237] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:40.947125 21225 catalog_manager.cc:5495] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 reported cstate change: term changed from 0 to 1, leader changed from <none> to 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130). New cstate: current_term: 1 leader_uuid: "726e3fcb13a840b39434fd349f5f0558" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } health_report { overall_health: HEALTHY } } }
I20260807 00:42:40.977361 21893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:40.974647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57216 (local address 127.1.116.130:41959)
0807 00:42:40.974738 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:40.974741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:40.974754 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:42:40.974863 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:40.974864 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:40.974886 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:40.974968 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:42:40.975151 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.975692 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.976487 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:40.976692 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:40.976774 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:40.976964 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:42:40.977084 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:42:40.977195 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:40.977213 (+ 18us) server_negotiation.cc:299] Negotiation successful
0807 00:42:40.977237 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:42:40.979558 21425 raft_consensus.cc:1261] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Refusing update from remote peer 726e3fcb13a840b39434fd349f5f0558: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:40.979558 21797 raft_consensus.cc:1261] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 1 FOLLOWER]: Refusing update from remote peer 726e3fcb13a840b39434fd349f5f0558: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:40.980063 21912 consensus_queue.cc:1035] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:40.980273 21908 consensus_queue.cc:1035] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:40.984206 21919 mvcc.cc:205] Tried to move back new op lower bound from 7315715526568448000 to 7315715526435987456. Current Snapshot: MvccSnapshot[applied={T|T < 7315715526568448000}]
I20260807 00:42:40.986135 21225 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:40.986268 21225 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:40.987257 21225 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Sending DeleteTablet for 3 replicas of tablet 8108980236184829a450823433bee975
I20260807 00:42:40.987624 21396 tablet_service.cc:1514] Processing DeleteTablet for tablet 8108980236184829a450823433bee975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:40 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:42776
I20260807 00:42:40.987696 21577 tablet_service.cc:1514] Processing DeleteTablet for tablet 8108980236184829a450823433bee975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:40 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:57208
I20260807 00:42:40.987792 21770 tablet_service.cc:1514] Processing DeleteTablet for tablet 8108980236184829a450823433bee975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:40 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:47478
I20260807 00:42:40.987923 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 21312
I20260807 00:42:40.987949 21929 tablet_replica.cc:330] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: stopping tablet replica
I20260807 00:42:40.988059 21932 tablet_replica.cc:330] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: stopping tablet replica
I20260807 00:42:40.988104 21929 raft_consensus.cc:2229] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:42:40.988197 21932 raft_consensus.cc:2229] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:42:40.988248 21929 raft_consensus.cc:2258] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:40.988329 21932 raft_consensus.cc:2258] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:41.000639 21932 ts_tablet_manager.cc:1905] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:41.000876 21933 tablet_replica.cc:330] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: stopping tablet replica
I20260807 00:42:41.001031 21933 raft_consensus.cc:2229] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:42:41.001200 21933 raft_consensus.cc:2258] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:41.002899 21932 ts_tablet_manager.cc:1918] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:41.002987 21932 log.cc:1196] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/8108980236184829a450823433bee975
I20260807 00:42:41.003252 21932 ts_tablet_manager.cc:1939] T 8108980236184829a450823433bee975 P 726e3fcb13a840b39434fd349f5f0558: Deleting consensus metadata
I20260807 00:42:41.003818 21199 catalog_manager.cc:4841] TS 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959): tablet 8108980236184829a450823433bee975 (table test-table [id=2d44a327fd2948aeb3d574a1ac60d290]) successfully deleted
I20260807 00:42:41.004082 21933 ts_tablet_manager.cc:1905] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:41.006083 21933 ts_tablet_manager.cc:1918] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:41.006167 21933 log.cc:1196] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/8108980236184829a450823433bee975
I20260807 00:42:41.006433 21933 ts_tablet_manager.cc:1939] T 8108980236184829a450823433bee975 P 60bbc58eee394409ad018c34418ca4c3: Deleting consensus metadata
I20260807 00:42:41.006971 21200 catalog_manager.cc:4841] TS 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405): tablet 8108980236184829a450823433bee975 (table test-table [id=2d44a327fd2948aeb3d574a1ac60d290]) successfully deleted
W20260807 00:42:41.008311 21201 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:33775 (error 108)
I20260807 00:42:41.008587 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 21495
I20260807 00:42:41.013320 21892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.008743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33775 (local address 127.0.0.1:42790)
0807 00:42:41.013252 (+ 4509us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.013291 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4488}
W20260807 00:42:41.016171 21201 catalog_manager.cc:4584] TS cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8108980236184829a450823433bee975: Network error: Client connection negotiation failed: client connection to 127.1.116.129:33775: connect: Connection refused (error 111)
I20260807 00:42:41.022055 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 21682
I20260807 00:42:41.035918 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 21173
I20260807 00:42:41.045804 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36905
--webserver_interface=127.1.116.190
--webserver_port=41219
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:41.137156 21937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:41.137480 21937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:41.137504 21937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:41.138748 21937 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:41.138794 21937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:41.138808 21937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:41.138821 21937 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:41.138834 21937 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:41.140257 21937 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=41219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:41.140447 21937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:41.140615 21937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:41.152194 21948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.152185 21957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.152194 21951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:41.152638 21937 server_base.cc:1034] running on GCE node
I20260807 00:42:41.153069 21937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:41.153363 21937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:41.154613 21937 hybrid_clock.cc:648] HybridClock initialized: now 1786063361154576 us; error 52 us; skew 500 ppm
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:41.157366 21937 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:41.157866 21937 webserver.cc:460] Webserver started at http://127.1.116.190:41219/ using document root <none> and password file <none>
I20260807 00:42:41.158149 21937 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:41.158239 21937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:41.160096 21937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:41.160864 21967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.161080 21937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:41.161185 21937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "830586726b8d4ecebdf880f66936be53"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "c4b6e95c4c4fb1987e33f31c55066eeb"
server_key_iv: "e04541638df2b20a64989558be8eb1ab"
server_key_version: "encryptionkey@0"
I20260807 00:42:41.161607 21937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:41.177407 21937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:41.179404 21937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:41.179569 21937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:41.184306 21937 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:36905
I20260807 00:42:41.184422 22041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:36905 every 8 connection(s)
I20260807 00:42:41.184779 21937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:41.186795 22042 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:42:41.187523 22042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Bootstrap starting.
I20260807 00:42:41.192775 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 21937
I20260807 00:42:41.193091 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:33775
--local_ip_for_outbound_sockets=127.1.116.129
--tserver_master_addrs=127.1.116.190:36905
--webserver_port=40951
--webserver_interface=127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:42:41.198787 22042 log.cc:824] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:41.201018 22042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:42:41.201330 22042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Bootstrap complete.
I20260807 00:42:41.204113 22042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.204386 22042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830586726b8d4ecebdf880f66936be53, State: Initialized, Role: FOLLOWER
I20260807 00:42:41.204563 22042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.204649 22042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:42:41.204689 22042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:42:41.204751 22042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:42:41.205927 22042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.206058 22042 leader_election.cc:302] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 830586726b8d4ecebdf880f66936be53; no voters:
I20260807 00:42:41.206291 22042 leader_election.cc:288] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:42:41.206403 22048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:42:41.206615 22048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 LEADER]: Becoming Leader. State: Replica: 830586726b8d4ecebdf880f66936be53, State: Running, Role: LEADER
I20260807 00:42:41.206682 22042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:41.206758 22048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.207134 22049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "830586726b8d4ecebdf880f66936be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } } }
I20260807 00:42:41.207252 22049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:41.207131 22048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 830586726b8d4ecebdf880f66936be53. Latest consensus state: current_term: 2 leader_uuid: "830586726b8d4ecebdf880f66936be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } } }
I20260807 00:42:41.207465 22048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:41.207576 22063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:41.208724 22063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:41.208833 22063 catalog_manager.cc:1259] Loaded cluster ID: 7a0071b36599428498c6e11fed1df519
I20260807 00:42:41.208869 22063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:42:41.209141 22063 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 00:42:41.213956 22063 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 00:42:41.216805 22063 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:42:41.217049 22063 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Loaded TSK: 0
I20260807 00:42:41.217299 22063 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:42:41.278993 22046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:41.279227 22046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:41.279268 22046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:41.280439 22046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:41.280489 22046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:41.280519 22046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:42:41.281953 22046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:33775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=40951
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:41.282196 22046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:41.282455 22046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:41.289072 22093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.289072 22097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.289114 22095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:41.289494 22046 server_base.cc:1034] running on GCE node
I20260807 00:42:41.289710 22046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:41.289961 22046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:41.291131 22046 hybrid_clock.cc:648] HybridClock initialized: now 1786063361291114 us; error 41 us; skew 500 ppm
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:41.293725 22046 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:42:41.294209 22046 webserver.cc:460] Webserver started at http://127.1.116.129:40951/ using document root <none> and password file <none>
I20260807 00:42:41.294448 22046 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:41.294507 22046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:41.295874 22046 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:41.296650 22106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.296916 22046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:41.297029 22046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "cccd6eece80e43b89de54997299a5a12"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "fb682f74a1b5763feb61665536419fd1"
server_key_iv: "18d0051c7b7c8750b70a19bb4f46c1bb"
server_key_version: "encryptionkey@0"
I20260807 00:42:41.297360 22046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:41.331223 22046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:41.333472 22046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:41.333638 22046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:41.333894 22046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:41.334399 22118 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:42:41.335844 22046 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:42:41.335903 22046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 00:42:41.335927 22046 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:42:41.336581 22046 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:42:41.336633 22046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:41.336669 22118 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Bootstrap starting.
I20260807 00:42:41.345582 22046 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:33775
I20260807 00:42:41.346073 22046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:42:41.346652 22290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:33775 every 8 connection(s)
I20260807 00:42:41.348307 22118 log.cc:824] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:41.349874 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 22046
I20260807 00:42:41.350281 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:41959
--local_ip_for_outbound_sockets=127.1.116.130
--tserver_master_addrs=127.1.116.190:36905
--webserver_port=34865
--webserver_interface=127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:42:41.354005 22118 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:42:41.354375 22118 tablet_bootstrap.cc:492] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Bootstrap complete.
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:41.355597 22118 ts_tablet_manager.cc:1397] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.015s
I20260807 00:42:41.357265 22118 raft_consensus.cc:348] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:41.357642 22118 raft_consensus.cc:729] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cccd6eece80e43b89de54997299a5a12, State: Initialized, Role: FOLLOWER
I20260807 00:42:41.357847 22118 consensus_queue.cc:260] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } } peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } }
I20260807 00:42:41.358259 22118 ts_tablet_manager.cc:1428] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:42:41.360198 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.347461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:48367 (local address 127.1.116.190:36905)
0807 00:42:41.348688 (+ 1227us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.348697 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.348742 (+ 45us) server_negotiation.cc:407] Connection header received
0807 00:42:41.348909 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.348915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.348967 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.349113 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:41.349778 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.350973 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.352013 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.352492 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.353083 (+ 591us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.355600 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:41.355605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:41.355606 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:41.355630 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:41.358027 (+ 2397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.358711 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.358715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.358715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.358825 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.359159 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.359160 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.359160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.359415 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:41.359494 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.359900 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.360032 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":126,"threads_started":1}
I20260807 00:42:41.360292 22295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.346956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:48367)
0807 00:42:41.347458 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.347508 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.348189 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.349181 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.349186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.349426 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:41.349617 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.349622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.351108 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.351109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.351893 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.351895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.352655 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.352657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.352808 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.353550 (+ 742us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:41.353569 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:41.355469 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:41.358187 (+ 2718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.358195 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.358198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.358610 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.358918 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.358919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.358919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.359073 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.359503 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:41.359511 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:41.359691 (+ 180us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.359828 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.360025 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":124,"threads_started":1}
I20260807 00:42:41.361189 22292 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36905
I20260807 00:42:41.361310 22292 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:41.361515 22292 heartbeater.cc:511] Master 127.1.116.190:36905 requested a full tablet report, sending...
I20260807 00:42:41.362428 21993 ts_manager.cc:194] Registered new tserver with Master: cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775)
I20260807 00:42:41.363695 21993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:48367
I20260807 00:42:41.364959 22292 heartbeater.cc:503] Master 127.1.116.190:36905 was elected leader, sending a full tablet report...
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:42:41.372260 22300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.363190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33775 (local address 127.0.0.1:42802)
0807 00:42:41.363507 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.363547 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.363642 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.363923 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.363928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.364349 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:41.364421 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.364424 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.365555 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.365556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.366172 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.366175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.366856 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.366857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.366920 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.367453 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:41.367458 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:41.369022 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:41.371081 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.371088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.371090 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.371302 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.371533 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.371533 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.371534 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.371590 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.371953 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:41.371960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:41.372007 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.372080 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.372183 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":87,"threads_started":1}
I20260807 00:42:41.372359 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.363281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42802 (local address 127.1.116.129:33775)
0807 00:42:41.363511 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.363516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.363645 (+ 129us) server_negotiation.cc:407] Connection header received
0807 00:42:41.363752 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.363755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.363774 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.363848 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:41.364512 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.365427 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.366267 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.366760 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.366834 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.369121 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:41.369129 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:41.369130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:41.369137 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:41.370986 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.371392 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.371395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.371396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.371457 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.371658 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.371659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.371659 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.371845 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:41.371947 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.372155 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.372260 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":113,"threads_started":1}
I20260807 00:42:41.372774 22156 tablet_service.cc:1514] Processing DeleteTablet for tablet 8108980236184829a450823433bee975 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:40 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:42802
I20260807 00:42:41.373417 22303 tablet_replica.cc:330] stopping tablet replica
I20260807 00:42:41.373564 22303 raft_consensus.cc:2229] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:42:41.373659 22303 raft_consensus.cc:2258] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:41.374038 22303 ts_tablet_manager.cc:1905] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:41.376158 22303 ts_tablet_manager.cc:1918] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:41.376271 22303 log.cc:1196] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/8108980236184829a450823433bee975
I20260807 00:42:41.376531 22303 ts_tablet_manager.cc:1939] T 8108980236184829a450823433bee975 P cccd6eece80e43b89de54997299a5a12: Deleting consensus metadata
I20260807 00:42:41.377146 21972 catalog_manager.cc:4841] TS cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775): tablet 8108980236184829a450823433bee975 (table test-table [id=2d44a327fd2948aeb3d574a1ac60d290]) successfully deleted
W20260807 00:42:41.441390 22298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:41.441622 22298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:41.441663 22298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:41.442811 22298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:41.442876 22298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:41.442922 22298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:42:41.444372 22298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:41959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=34865
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:41.444634 22298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:41.444864 22298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:41.450248 22317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.450289 22309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.450294 22313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:41.450503 22298 server_base.cc:1034] running on GCE node
I20260807 00:42:41.450762 22298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:41.451053 22298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:41.452255 22298 hybrid_clock.cc:648] HybridClock initialized: now 1786063361452229 us; error 48 us; skew 500 ppm
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:41.454896 22298 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:42:41.455370 22298 webserver.cc:460] Webserver started at http://127.1.116.130:34865/ using document root <none> and password file <none>
I20260807 00:42:41.455631 22298 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:41.455716 22298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:41.457113 22298 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:42:41.457782 22326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.457999 22298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:41.458113 22298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "726e3fcb13a840b39434fd349f5f0558"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "03e5af14091c2fca59d9fbbd81532dd4"
server_key_iv: "076353fa69c5917689be2d016c0f0d9f"
server_key_version: "encryptionkey@0"
I20260807 00:42:41.458431 22298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:41.476249 22298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:41.478493 22298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:41.478693 22298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:41.478874 22298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:41.479133 22298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:41.479171 22298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.479192 22298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:41.479206 22298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.487247 22298 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:41959
I20260807 00:42:41.487317 22485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:41959 every 8 connection(s)
I20260807 00:42:41.487643 22298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:42:41.489212 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 22298
I20260807 00:42:41.489593 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:40405
--local_ip_for_outbound_sockets=127.1.116.131
--tserver_master_addrs=127.1.116.190:36905
--webserver_port=40953
--webserver_interface=127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:41.500259 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.488552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:39811 (local address 127.1.116.190:36905)
0807 00:42:41.488660 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.488663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.488877 (+ 214us) server_negotiation.cc:407] Connection header received
0807 00:42:41.489819 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.489821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.489842 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.489876 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:41.491536 (+ 1660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.492227 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.493361 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.493656 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.493731 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.496513 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:41.496515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:41.496516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:41.496525 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:41.498451 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.498994 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.498995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.498995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.499064 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.499356 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.499357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.499357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.499562 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:41.499651 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.499974 (+ 323us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.500114 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:42:41.500370 22488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.488503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:39811)
0807 00:42:41.488720 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.488769 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.489657 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.490088 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.490093 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.491067 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:41.491399 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.491405 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.492351 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.492352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.493245 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.493247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.493753 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.493754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.493871 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.494515 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:41.494564 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:41.496388 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:41.498603 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.498613 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.498620 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.498896 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.499150 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.499151 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.499151 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.499259 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.499645 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:41.499650 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:41.499803 (+ 153us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.499893 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.500046 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":105,"threads_started":1}
I20260807 00:42:41.501075 22486 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36905
I20260807 00:42:41.501368 22486 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:41.501596 22486 heartbeater.cc:511] Master 127.1.116.190:36905 requested a full tablet report, sending...
I20260807 00:42:41.502086 21993 ts_manager.cc:194] Registered new tserver with Master: 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959)
I20260807 00:42:41.502691 21993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:39811
W20260807 00:42:41.576077 22489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:41.576267 22489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:41.576289 22489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:41.577318 22489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:41.577356 22489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:41.577382 22489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:42:41.578806 22489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:40405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=40953
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:41.578986 22489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:41.579151 22489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:41.584200 22498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.584319 22494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.584337 22495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:41.584625 22489 server_base.cc:1034] running on GCE node
I20260807 00:42:41.584861 22489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:41.585105 22489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:41.586331 22489 hybrid_clock.cc:648] HybridClock initialized: now 1786063361586303 us; error 34 us; skew 500 ppm
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:41.589208 22489 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:42:41.589699 22489 webserver.cc:460] Webserver started at http://127.1.116.131:40953/ using document root <none> and password file <none>
I20260807 00:42:41.589901 22489 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:41.589965 22489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:41.591451 22489 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:41.592172 22508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.592446 22489 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:42:41.592563 22489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "60bbc58eee394409ad018c34418ca4c3"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "82352960d86dd291eab09099bd4266a1"
server_key_iv: "447e45b63e0a6ee9d3d65b3b99ba1977"
server_key_version: "encryptionkey@0"
I20260807 00:42:41.592984 22489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:41.607836 22489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:41.610606 22489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:41.610811 22489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:41.611133 22489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:41.611554 22489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:41.611596 22489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.611645 22489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:41.611682 22489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.621397 22489 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:40405
I20260807 00:42:41.621450 22659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:40405 every 8 connection(s)
I20260807 00:42:41.621846 22489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:42:41.627687 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 22489
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:41.635478 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.622826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:38915 (local address 127.1.116.190:36905)
0807 00:42:41.622930 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.622933 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.623227 (+ 294us) server_negotiation.cc:407] Connection header received
0807 00:42:41.624076 (+ 849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.624077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.624089 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.624149 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:41.625543 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.626081 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.627943 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.628285 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.628362 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.631851 (+ 3489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:41.631853 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:41.631853 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:41.631861 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:41.633669 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.634222 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.634223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.634224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.634284 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.634645 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.634646 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.634647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.634792 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:41.634879 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.635309 (+ 430us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.635392 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 00:42:41.635747 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.622807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:38915)
0807 00:42:41.623052 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.623105 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.623949 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.624254 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.624259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.625118 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:41.625428 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.625435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.626186 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.626187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.627768 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.627772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.628428 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.628430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.628651 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.629350 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:41.629359 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:41.631757 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:41.633737 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.633744 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.633746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.634101 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.634377 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:41.634378 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:41.634379 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:41.634503 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:41.634899 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:41.634907 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:41.635108 (+ 201us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.635239 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.635449 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":113,"threads_started":1}
I20260807 00:42:41.636265 22660 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36905
I20260807 00:42:41.636379 22660 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:41.636601 22660 heartbeater.cc:511] Master 127.1.116.190:36905 requested a full tablet report, sending...
I20260807 00:42:41.637018 21993 ts_manager.cc:194] Registered new tserver with Master: 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405)
I20260807 00:42:41.637562 21993 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:38915
I20260807 00:42:41.637943 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.628250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37146 (local address 127.1.116.190:36905)
0807 00:42:41.628454 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.628456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.628550 (+ 94us) server_negotiation.cc:407] Connection header received
0807 00:42:41.628613 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.628614 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.628628 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.628709 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:41.629717 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.630367 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.631307 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.631591 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.631667 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.633381 (+ 1714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:41.633383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:41.633383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:41.633401 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:41.635284 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.636955 (+ 1671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.636956 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.636957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.637025 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:41.637324 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:41.637325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:41.637325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:41.637453 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:41.637703 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.637750 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.637838 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":102,"threads_started":1}
I20260807 00:42:41.639076 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:42:41.642704 22666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.639525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37158 (local address 127.1.116.190:36905)
0807 00:42:41.639583 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.639586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.639669 (+ 83us) server_negotiation.cc:407] Connection header received
0807 00:42:41.639756 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.639757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.639802 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.639904 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:42:41.640007 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.640891 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.641662 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.642057 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.642142 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.642288 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:42:41.642413 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:42:41.642505 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.642564 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.642593 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:42:41.642993 21993 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:37158:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:42:41.646116 21993 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:42:41.648957 22156 tablet_service.cc:1467] Processing CreateTablet for tablet 10dc49eea9e647fc8bc6ba286c5429db (DEFAULT_TABLE table=test-table [id=d27759f71d124e22a68abf78b4352b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:41.649263 22156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dc49eea9e647fc8bc6ba286c5429db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:41.651279 22118 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Bootstrap starting.
I20260807 00:42:41.651976 22118 tablet_bootstrap.cc:654] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:41.652834 22118 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: No bootstrap required, opened a new log
I20260807 00:42:41.652894 22118 ts_tablet_manager.cc:1397] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260807 00:42:41.653083 22118 raft_consensus.cc:348] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.653180 22118 raft_consensus.cc:374] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:41.653232 22118 raft_consensus.cc:729] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cccd6eece80e43b89de54997299a5a12, State: Initialized, Role: FOLLOWER
I20260807 00:42:41.653280 22118 consensus_queue.cc:260] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.653379 22118 ts_tablet_manager.cc:1428] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260807 00:42:41.654227 22300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.648548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:40405 (local address 127.0.0.1:47486)
0807 00:42:41.648708 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.648719 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.648757 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.649634 (+ 877us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.649635 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.649637 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:41.649699 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.649700 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.651092 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.651093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.652381 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.652385 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.653915 (+ 1530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.653916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.653997 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.654019 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.654054 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.654121 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 00:42:41.654889 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.648617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47486 (local address 127.1.116.131:40405)
0807 00:42:41.649054 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.649060 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.649178 (+ 118us) server_negotiation.cc:407] Connection header received
0807 00:42:41.649471 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.649474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.649496 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.649770 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:41.649844 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.650943 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.652683 (+ 1740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.653816 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.654631 (+ 815us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.654718 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.654747 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.654798 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":96,"threads_started":1}
I20260807 00:42:41.655553 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.648548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:41959 (local address 127.0.0.1:57218)
0807 00:42:41.649011 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.649025 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.649085 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.650864 (+ 1779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.650866 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.650868 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:41.650922 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.650924 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.652496 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.652498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.653897 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.653900 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.655276 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.655277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.655360 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.655366 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.655398 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.655451 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":260,"thread_start_us":121,"threads_started":1}
I20260807 00:42:41.655946 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.648715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57218 (local address 127.1.116.130:41959)
0807 00:42:41.650663 (+ 1948us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.650668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.650681 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:42:41.650727 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.650730 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.650747 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.651011 (+ 264us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:41.651148 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.652353 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.654213 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.655158 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.655679 (+ 521us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.655750 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.655780 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.655830 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1907,"thread_start_us":99,"threads_started":1}
I20260807 00:42:41.655849 22569 tablet_service.cc:1467] Processing CreateTablet for tablet 10dc49eea9e647fc8bc6ba286c5429db (DEFAULT_TABLE table=test-table [id=d27759f71d124e22a68abf78b4352b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:41.656266 22569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dc49eea9e647fc8bc6ba286c5429db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:41.656770 22396 tablet_service.cc:1467] Processing CreateTablet for tablet 10dc49eea9e647fc8bc6ba286c5429db (DEFAULT_TABLE table=test-table [id=d27759f71d124e22a68abf78b4352b1b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:41.657162 22396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10dc49eea9e647fc8bc6ba286c5429db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:41.659346 22672 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Bootstrap starting.
I20260807 00:42:41.660089 22672 tablet_bootstrap.cc:654] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:41.660797 22672 log.cc:824] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:41.661669 22674 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Bootstrap starting.
I20260807 00:42:41.661798 22672 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: No bootstrap required, opened a new log
I20260807 00:42:41.661923 22672 ts_tablet_manager.cc:1397] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:42:41.662619 22674 tablet_bootstrap.cc:654] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:41.663259 22674 log.cc:824] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:41.664283 22674 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: No bootstrap required, opened a new log
I20260807 00:42:41.664364 22674 ts_tablet_manager.cc:1397] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 00:42:41.664422 22672 raft_consensus.cc:348] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.664638 22672 raft_consensus.cc:374] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:41.664691 22672 raft_consensus.cc:729] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60bbc58eee394409ad018c34418ca4c3, State: Initialized, Role: FOLLOWER
I20260807 00:42:41.664877 22672 consensus_queue.cc:260] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.665225 22672 ts_tablet_manager.cc:1428] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:42:41.665270 22660 heartbeater.cc:503] Master 127.1.116.190:36905 was elected leader, sending a full tablet report...
I20260807 00:42:41.666931 22674 raft_consensus.cc:348] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.667071 22674 raft_consensus.cc:374] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:41.667093 22674 raft_consensus.cc:729] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 726e3fcb13a840b39434fd349f5f0558, State: Initialized, Role: FOLLOWER
I20260807 00:42:41.667224 22674 consensus_queue.cc:260] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.667536 22674 ts_tablet_manager.cc:1428] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260807 00:42:41.667629 22486 heartbeater.cc:503] Master 127.1.116.190:36905 was elected leader, sending a full tablet report...
I20260807 00:42:41.692210 22679 raft_consensus.cc:482] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:41.692371 22679 raft_consensus.cc:504] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.692958 22679 leader_election.cc:288] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959), cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775)
I20260807 00:42:41.696920 22663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.693142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33775 (local address 127.1.116.131:56685)
0807 00:42:41.693286 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.693303 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.693370 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.693675 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.693676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.693679 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:41.693784 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.693785 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.694622 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.694623 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.695758 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.695786 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.696621 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.696622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.696704 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.696708 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.696778 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.696830 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260807 00:42:41.696920 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.693406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:56685 (local address 127.1.116.129:33775)
0807 00:42:41.693475 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.693477 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.693488 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:42:41.693523 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.693524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.693560 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.693659 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:41.693861 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.694494 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.695884 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.696538 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.696627 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.696713 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.696786 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.696832 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 00:42:41.697542 22669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.693322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:38971 (local address 127.1.116.130:41959)
0807 00:42:41.693473 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.693476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.693488 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:42:41.693568 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.693569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.693580 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.693671 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:41.694044 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.694930 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.696259 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.697082 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.697176 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.697186 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.697371 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.697442 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260807 00:42:41.697611 22214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc49eea9e647fc8bc6ba286c5429db" candidate_uuid: "60bbc58eee394409ad018c34418ca4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cccd6eece80e43b89de54997299a5a12" is_pre_election: true
I20260807 00:42:41.697782 22682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.693142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:41959 (local address 127.1.116.131:38971)
0807 00:42:41.693409 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.693423 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:41.693496 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:41.693880 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:41.693881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:41.693883 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:41.693957 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.693959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.695037 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.695047 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.696164 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:41.696167 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.697186 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.697187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:41.697265 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.697269 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:42:41.697647 (+ 378us) client_negotiation.cc:240] Negotiation successful
0807 00:42:41.697697 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":18,"thread_start_us":160,"threads_started":1}
I20260807 00:42:41.697822 22214 raft_consensus.cc:2454] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60bbc58eee394409ad018c34418ca4c3 in term 0.
I20260807 00:42:41.698236 22512 leader_election.cc:302] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60bbc58eee394409ad018c34418ca4c3, cccd6eece80e43b89de54997299a5a12; no voters:
I20260807 00:42:41.698191 22422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc49eea9e647fc8bc6ba286c5429db" candidate_uuid: "60bbc58eee394409ad018c34418ca4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "726e3fcb13a840b39434fd349f5f0558" is_pre_election: true
I20260807 00:42:41.698374 22422 raft_consensus.cc:2454] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60bbc58eee394409ad018c34418ca4c3 in term 0.
I20260807 00:42:41.698443 22679 raft_consensus.cc:2781] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:42:41.698549 22679 raft_consensus.cc:482] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:42:41.698586 22679 raft_consensus.cc:3037] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:41.699823 22679 raft_consensus.cc:504] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.700008 22679 leader_election.cc:288] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [CANDIDATE]: Term 1 election: Requested vote from peers 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959), cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775)
I20260807 00:42:41.700196 22422 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc49eea9e647fc8bc6ba286c5429db" candidate_uuid: "60bbc58eee394409ad018c34418ca4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "726e3fcb13a840b39434fd349f5f0558"
I20260807 00:42:41.700261 22214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10dc49eea9e647fc8bc6ba286c5429db" candidate_uuid: "60bbc58eee394409ad018c34418ca4c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cccd6eece80e43b89de54997299a5a12"
I20260807 00:42:41.700326 22422 raft_consensus.cc:3037] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:41.700338 22214 raft_consensus.cc:3037] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:41.701455 22422 raft_consensus.cc:2454] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60bbc58eee394409ad018c34418ca4c3 in term 1.
I20260807 00:42:41.701455 22214 raft_consensus.cc:2454] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60bbc58eee394409ad018c34418ca4c3 in term 1.
I20260807 00:42:41.701676 22510 leader_election.cc:302] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 60bbc58eee394409ad018c34418ca4c3, 726e3fcb13a840b39434fd349f5f0558; no voters:
I20260807 00:42:41.701829 22679 raft_consensus.cc:2781] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:41.702081 22679 raft_consensus.cc:686] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 1 LEADER]: Becoming Leader. State: Replica: 60bbc58eee394409ad018c34418ca4c3, State: Running, Role: LEADER
I20260807 00:42:41.702196 22679 consensus_queue.cc:237] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:41.703414 21992 catalog_manager.cc:5495] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131). New cstate: current_term: 1 leader_uuid: "60bbc58eee394409ad018c34418ca4c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } health_report { overall_health: UNKNOWN } } }
I20260807 00:42:41.720075 22668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.717485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47500 (local address 127.1.116.131:40405)
0807 00:42:41.717560 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:41.717562 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:41.717575 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:42:41.717680 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:41.717681 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:41.717701 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:41.717784 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:42:41.717901 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.718361 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.719216 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:41.719423 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:41.719492 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:41.719612 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:42:41.719714 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:42:41.719778 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:41.719943 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 00:42:41.719971 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:42:41.722455 22214 raft_consensus.cc:1261] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Refusing update from remote peer 60bbc58eee394409ad018c34418ca4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:41.722518 22422 raft_consensus.cc:1261] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 1 FOLLOWER]: Refusing update from remote peer 60bbc58eee394409ad018c34418ca4c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:41.722921 22683 consensus_queue.cc:1035] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:41.723075 22679 consensus_queue.cc:1035] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:41.726001 22686 mvcc.cc:205] Tried to move back new op lower bound from 7315715529611907072 to 7315715529533853696. Current Snapshot: MvccSnapshot[applied={T|T < 7315715529611907072}]
I20260807 00:42:41.726402 22687 mvcc.cc:205] Tried to move back new op lower bound from 7315715529611907072 to 7315715529533853696. Current Snapshot: MvccSnapshot[applied={T|T < 7315715529611907072}]
I20260807 00:42:41.727290 22688 mvcc.cc:205] Tried to move back new op lower bound from 7315715529611907072 to 7315715529533853696. Current Snapshot: MvccSnapshot[applied={T|T < 7315715529611907072 or (T in {7315715529611907072})}]
I20260807 00:42:41.728886 21993 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:37158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:41.729010 21993 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:37158:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:41.729904 21993 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Sending DeleteTablet for 3 replicas of tablet 10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:41.730310 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 22046
I20260807 00:42:41.730286 22569 tablet_service.cc:1514] Processing DeleteTablet for tablet 10dc49eea9e647fc8bc6ba286c5429db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:41 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:47486
I20260807 00:42:41.730419 22396 tablet_service.cc:1514] Processing DeleteTablet for tablet 10dc49eea9e647fc8bc6ba286c5429db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:41 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:57218
I20260807 00:42:41.737771 22700 tablet_replica.cc:330] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: stopping tablet replica
I20260807 00:42:41.737952 22700 raft_consensus.cc:2229] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:42:41.738129 22700 raft_consensus.cc:2258] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:41.738329 22699 tablet_replica.cc:330] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: stopping tablet replica
I20260807 00:42:41.738449 22699 raft_consensus.cc:2229] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:42:41.738620 22699 raft_consensus.cc:2258] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:41.739034 22700 ts_tablet_manager.cc:1905] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:41.739162 22699 ts_tablet_manager.cc:1905] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:41.740670 22699 ts_tablet_manager.cc:1918] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:41.740780 22699 log.cc:1196] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:41.741086 22699 ts_tablet_manager.cc:1939] T 10dc49eea9e647fc8bc6ba286c5429db P 60bbc58eee394409ad018c34418ca4c3: Deleting consensus metadata
I20260807 00:42:41.741214 22700 ts_tablet_manager.cc:1918] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:41.741314 22700 log.cc:1196] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:41.741618 22700 ts_tablet_manager.cc:1939] T 10dc49eea9e647fc8bc6ba286c5429db P 726e3fcb13a840b39434fd349f5f0558: Deleting consensus metadata
I20260807 00:42:41.741616 21970 catalog_manager.cc:4841] TS 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405): tablet 10dc49eea9e647fc8bc6ba286c5429db (table test-table [id=d27759f71d124e22a68abf78b4352b1b]) successfully deleted
I20260807 00:42:41.742070 21969 catalog_manager.cc:4841] TS 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959): tablet 10dc49eea9e647fc8bc6ba286c5429db (table test-table [id=d27759f71d124e22a68abf78b4352b1b]) successfully deleted
W20260807 00:42:41.743656 21972 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:33775 (error 108)
I20260807 00:42:41.743995 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 22298
I20260807 00:42:41.748402 22670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:41.744112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:33775 (local address 127.0.0.1:42804)
0807 00:42:41.748293 (+ 4181us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:41.748359 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:33775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4148}
W20260807 00:42:41.754405 21972 catalog_manager.cc:4584] TS cccd6eece80e43b89de54997299a5a12 (127.1.116.129:33775): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 10dc49eea9e647fc8bc6ba286c5429db: Network error: Client connection negotiation failed: client connection to 127.1.116.129:33775: connect: Connection refused (error 111)
I20260807 00:42:41.757122 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 22489
I20260807 00:42:41.767166 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 21937
I20260807 00:42:41.777827 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36905
--webserver_interface=127.1.116.190
--webserver_port=41219
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:41.861958 22704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:41.862151 22704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:41.862174 22704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:41.863320 22704 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:41.863368 22704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:41.863386 22704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:41.863399 22704 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:41.863413 22704 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:41.864826 22704 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=41219
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:41.865011 22704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:41.865173 22704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:41.870641 22712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:41.870729 22704 server_base.cc:1034] running on GCE node
W20260807 00:42:41.870651 22713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:41.870667 22715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:41.871201 22704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:41.871595 22704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:41.872773 22704 hybrid_clock.cc:648] HybridClock initialized: now 1786063361872755 us; error 53 us; skew 500 ppm
Aug 07 00:42:41 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063361, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:41.875638 22704 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:41.876139 22704 webserver.cc:460] Webserver started at http://127.1.116.190:41219/ using document root <none> and password file <none>
I20260807 00:42:41.876413 22704 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:41.876488 22704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:41.877939 22704 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:41.878769 22726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:41.878979 22704 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:41.879101 22704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "830586726b8d4ecebdf880f66936be53"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "c4b6e95c4c4fb1987e33f31c55066eeb"
server_key_iv: "e04541638df2b20a64989558be8eb1ab"
server_key_version: "encryptionkey@0"
I20260807 00:42:41.879494 22704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:41.895857 22704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:41.898070 22704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:41.898262 22704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:41.903303 22704 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:36905
I20260807 00:42:41.903357 22801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:36905 every 8 connection(s)
I20260807 00:42:41.903824 22704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:41.905515 22802 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:42:41.906230 22802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Bootstrap starting.
I20260807 00:42:41.914080 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 22704
I20260807 00:42:41.914459 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:33775
--local_ip_for_outbound_sockets=127.1.116.129
--tserver_master_addrs=127.1.116.190:36905
--webserver_port=40951
--webserver_interface=127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:42:41.915853 22802 log.cc:824] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:41.918515 22802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:42:41.918977 22802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53: Bootstrap complete.
I20260807 00:42:41.922070 22802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.922336 22802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830586726b8d4ecebdf880f66936be53, State: Initialized, Role: FOLLOWER
I20260807 00:42:41.922605 22802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.922711 22802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:42:41.922770 22802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:42:41.922817 22802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 2 FOLLOWER]: Advancing to term 3
I20260807 00:42:41.923859 22802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.923993 22802 leader_election.cc:302] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 830586726b8d4ecebdf880f66936be53; no voters:
I20260807 00:42:41.924225 22802 leader_election.cc:288] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 00:42:41.924263 22807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 00:42:41.924419 22807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [term 3 LEADER]: Becoming Leader. State: Replica: 830586726b8d4ecebdf880f66936be53, State: Running, Role: LEADER
I20260807 00:42:41.924535 22802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:41.924527 22807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } }
I20260807 00:42:41.925014 22807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 830586726b8d4ecebdf880f66936be53. Latest consensus state: current_term: 3 leader_uuid: "830586726b8d4ecebdf880f66936be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } } }
I20260807 00:42:41.925155 22807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:41.925014 22808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "830586726b8d4ecebdf880f66936be53" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830586726b8d4ecebdf880f66936be53" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36905 } } }
I20260807 00:42:41.925405 22808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830586726b8d4ecebdf880f66936be53 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:41.925737 22817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:41.926909 22817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:41.927062 22817 catalog_manager.cc:1259] Loaded cluster ID: 7a0071b36599428498c6e11fed1df519
I20260807 00:42:41.927119 22817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 00:42:41.927768 22817 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fad7de869ad google::LogMessage::Fail() at ??:0
@ 0x7fad7de8988a google::LogMessage::SendToLog() at ??:0
@ 0x7fad7de864de google::LogMessage::Flush() at ??:0
@ 0x7fad7de87a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fad85da70f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fad85db08b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fad7fd540d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fad7fd49954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fad80bad6db start_thread at ??:0
@ 0x7fad7c98461f clone at ??:0
W20260807 00:42:42.026288 22806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:42.026587 22806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:42.026631 22806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:42.027953 22806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:42.028024 22806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:42.028092 22806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:42:42.029822 22806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:33775
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=40951
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22806
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:42.030154 22806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:42.030448 22806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:42.036500 22858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:42.036623 22860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:42.036540 22857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:42.036926 22806 server_base.cc:1034] running on GCE node
I20260807 00:42:42.037201 22806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:42.037552 22806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:42.038757 22806 hybrid_clock.cc:648] HybridClock initialized: now 1786063362038741 us; error 32 us; skew 500 ppm
Aug 07 00:42:42 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063362, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:42.042419 22806 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:42:42.043020 22806 webserver.cc:460] Webserver started at http://127.1.116.129:40951/ using document root <none> and password file <none>
I20260807 00:42:42.043303 22806 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:42.043377 22806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:42.044963 22806 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:42:42.045874 22868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:42.046192 22806 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:42:42.046320 22806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "cccd6eece80e43b89de54997299a5a12"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "fb682f74a1b5763feb61665536419fd1"
server_key_iv: "18d0051c7b7c8750b70a19bb4f46c1bb"
server_key_version: "encryptionkey@0"
I20260807 00:42:42.046838 22806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:42.073988 22806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:42.076716 22806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:42.076905 22806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:42.077194 22806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:42.077762 22877 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:42:42.080030 22806 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:42:42.080111 22806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260807 00:42:42.080161 22806 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:42:42.081001 22806 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:42:42.081064 22806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:42.081221 22877 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Bootstrap starting.
I20260807 00:42:42.089691 22806 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:33775
I20260807 00:42:42.089869 23030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:33775 every 8 connection(s)
I20260807 00:42:42.090185 22806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:42:42.091421 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 22806
I20260807 00:42:42.091789 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:41959
--local_ip_for_outbound_sockets=127.1.116.130
--tserver_master_addrs=127.1.116.190:36905
--webserver_port=34865
--webserver_interface=127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:42:42.093504 22877 log.cc:824] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:42.096174 22877 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:42:42.096498 22877 tablet_bootstrap.cc:492] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Bootstrap complete.
I20260807 00:42:42.097560 22877 ts_tablet_manager.cc:1397] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.004s
I20260807 00:42:42.098884 22877 raft_consensus.cc:348] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:42.099176 22877 raft_consensus.cc:729] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cccd6eece80e43b89de54997299a5a12, State: Initialized, Role: FOLLOWER
I20260807 00:42:42.099354 22877 consensus_queue.cc:260] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:42.099702 22877 ts_tablet_manager.cc:1428] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
W20260807 00:42:42.158577 23034 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:42:42.091079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:48417)
0807 00:42:42.091359 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.091419 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:42.092649 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:42.158325 (+ 65676us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":151,"threads_started":1}
W20260807 00:42:42.158946 23031 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:36905 (0 consecutive failures): Network error: Failed to ping master at 127.1.116.190:36905: Client connection negotiation failed: client connection to 127.1.116.190:36905: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:42:42.159418 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.159247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:50035)
0807 00:42:42.159325 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.159387 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 00:42:42.159852 23034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.159718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:57647)
0807 00:42:42.159801 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.159832 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 00:42:42.160012 23031 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:42:42.185330 23037 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:42.185516 23037 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:42.185540 23037 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:42.186698 23037 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:42.186741 23037 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:42.186769 23037 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:42:42.188270 23037 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:41959
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=34865
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23037
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:42.188509 23037 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:42.188731 23037 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:42.193786 23046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:42.193825 23048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:42.194105 23037 server_base.cc:1034] running on GCE node
W20260807 00:42:42.193842 23050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:42.194483 23037 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:42.194821 23037 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:42.195997 23037 hybrid_clock.cc:648] HybridClock initialized: now 1786063362195984 us; error 48 us; skew 500 ppm
Aug 07 00:42:42 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063362, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:42.198887 23037 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:42:42.199395 23037 webserver.cc:460] Webserver started at http://127.1.116.130:34865/ using document root <none> and password file <none>
I20260807 00:42:42.199661 23037 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:42.199713 23037 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:42.201299 23037 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:42.202011 23059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:42.202240 23037 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:42:42.202342 23037 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "726e3fcb13a840b39434fd349f5f0558"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "03e5af14091c2fca59d9fbbd81532dd4"
server_key_iv: "076353fa69c5917689be2d016c0f0d9f"
server_key_version: "encryptionkey@0"
I20260807 00:42:42.202720 23037 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:42.243741 23037 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:42.245728 23037 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:42.245839 23037 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:42.246028 23037 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:42.246335 23037 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:42.246377 23037 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:42.246398 23037 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:42.246412 23037 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:42.253839 23037 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:41959
I20260807 00:42:42.254035 23215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:41959 every 8 connection(s)
I20260807 00:42:42.254282 23037 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:42:42.255585 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.255125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:52989)
0807 00:42:42.255313 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.255405 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
W20260807 00:42:42.256130 23218 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:36905 (0 consecutive failures): Network error: Failed to ping master at 127.1.116.190:36905: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
I20260807 00:42:42.256481 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.256378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:34597)
0807 00:42:42.256433 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.256463 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 00:42:42.256793 23222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.256708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:60949)
0807 00:42:42.256756 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.256778 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 00:42:42.256961 23218 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:42:42.261296 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 23037
I20260807 00:42:42.261550 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:40405
--local_ip_for_outbound_sockets=127.1.116.131
--tserver_master_addrs=127.1.116.190:36905
--webserver_port=40953
--webserver_interface=127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:42.348835 23223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:42.349047 23223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:42.349074 23223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:42.350580 23223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:42.350641 23223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:42.350672 23223 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:42:42.352239 23223 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:40405
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=40953
--tserver_master_addrs=127.1.116.190:36905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:42.352475 23223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:42.352738 23223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:42.358299 23229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:42.358322 23233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:42.358518 23223 server_base.cc:1034] running on GCE node
W20260807 00:42:42.358363 23230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:42.358835 23223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:42.359045 23223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:42.360255 23223 hybrid_clock.cc:648] HybridClock initialized: now 1786063362360259 us; error 67 us; skew 500 ppm
Aug 07 00:42:42 dist-test-slave-h6n0 krb5kdc[21156](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063362, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:42.363569 23223 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:42:42.364069 23223 webserver.cc:460] Webserver started at http://127.1.116.131:40953/ using document root <none> and password file <none>
I20260807 00:42:42.364305 23223 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:42.364356 23223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:42.365710 23223 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:42.366429 23245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:42.366662 23223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:42.366771 23223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "60bbc58eee394409ad018c34418ca4c3"
format_stamp: "Formatted at 2026-08-07 00:42:40 on dist-test-slave-h6n0"
server_key: "82352960d86dd291eab09099bd4266a1"
server_key_iv: "447e45b63e0a6ee9d3d65b3b99ba1977"
server_key_version: "encryptionkey@0"
I20260807 00:42:42.367121 23223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:42.386672 23223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:42.389097 23223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:42.389284 23223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:42.389570 23223 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:42.389919 23223 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:42.389982 23223 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:42.390023 23223 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:42.390071 23223 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:42.397745 23223 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:40405
I20260807 00:42:42.397816 23408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:40405 every 8 connection(s)
I20260807 00:42:42.398147 23223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:42:42.399430 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.398850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:52443)
0807 00:42:42.399070 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.399219 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":80,"threads_started":1}
W20260807 00:42:42.399916 23409 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:36905 (0 consecutive failures): Network error: Failed to ping master at 127.1.116.190:36905: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
I20260807 00:42:42.400277 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.400170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:49403)
0807 00:42:42.400224 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.400258 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 00:42:42.400563 23413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:42.400475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:40879)
0807 00:42:42.400518 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:42.400540 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 00:42:42.400668 23409 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:42:42.408435 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 23223
I20260807 00:42:43.160809 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:43.160449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:44767)
0807 00:42:43.160653 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:43.160697 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":107,"threads_started":1}
I20260807 00:42:43.257663 23417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:43.257376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:35197)
0807 00:42:43.257549 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:43.257596 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260807 00:42:43.401330 23418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:43.401087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:55007)
0807 00:42:43.401228 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:43.401273 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":65,"threads_started":1}
I20260807 00:42:43.618099 23419 raft_consensus.cc:482] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:43.618283 23419 raft_consensus.cc:504] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:43.618814 23419 leader_election.cc:288] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405), 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959)
Aug 07 00:42:43 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063362, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.130@KRBTEST.COM
Aug 07 00:42:43 dist-test-slave-h6n0 krb5kdc[21156](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063362, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.131@KRBTEST.COM
I20260807 00:42:43.632663 23424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:43.619258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:35117 (local address 127.1.116.130:41959)
0807 00:42:43.619421 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:43.619437 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:43.619482 (+ 45us) server_negotiation.cc:407] Connection header received
0807 00:42:43.619987 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:43.619990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:43.620036 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:43.620150 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:43.621133 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.621992 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.623750 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.624396 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.624527 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:43.627959 (+ 3432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:43.627963 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:43.627963 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:43.627974 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:43.630116 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:43.630938 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:43.630942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:43.630942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:43.631074 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:43.631542 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:43.631544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:43.631544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:43.631873 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:43.631973 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:43.632401 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 00:42:43.632546 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260807 00:42:43.632663 23422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:43.619203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:44913 (local address 127.1.116.131:40405)
0807 00:42:43.619459 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:43.619471 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:43.619511 (+ 40us) server_negotiation.cc:407] Connection header received
0807 00:42:43.620092 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:43.620095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:43.620191 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:43.620377 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:43.621108 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.622192 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.623755 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.624396 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.624527 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:43.628263 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:43.628266 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:43.628267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:43.628277 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:43.630182 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:43.630938 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:43.630942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:43.630942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:43.631074 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:43.631542 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:43.631544 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:43.631544 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:43.631873 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:43.631973 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:43.632401 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 00:42:43.632546 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":65,"threads_started":1}
I20260807 00:42:43.632665 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:43.619009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:41959 (local address 127.1.116.129:35117)
0807 00:42:43.619172 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:43.619186 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:43.619250 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:43.620272 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:43.620276 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:43.620757 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:43.620968 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:43.620976 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.622108 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.622109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:43.623634 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:43.623638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.624521 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.624522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:43.624676 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:43.625546 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:43.625551 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:43.627811 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:43.630239 (+ 2428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:43.630248 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:43.630251 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:43.630831 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:43.631194 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:43.631195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:43.631195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:43.631406 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:43.631971 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:43.631979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:43.632165 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 00:42:43.632295 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 00:42:43.632528 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":54,"thread_start_us":152,"threads_started":1}
I20260807 00:42:43.632733 23416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:43.618974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:40405 (local address 127.1.116.129:44913)
0807 00:42:43.619077 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:43.619088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:43.619351 (+ 263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:43.620361 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:43.620363 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:43.620747 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:43.620968 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:43.620974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.622327 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.622328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:43.623634 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:43.623638 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:43.624520 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:43.624522 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:43.624676 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:43.625481 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:43.625520 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:43.628148 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:43.630267 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:43.630268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:43.630268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:43.630831 (+ 563us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:43.631194 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:43.631195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:43.631195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:43.631406 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:43.631971 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:43.631979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:43.632165 (+ 186us) client_negotiation.cc:769] Sending connection context
0807 00:42:43.632295 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 00:42:43.632593 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":291}
W20260807 00:42:43.633756 22872 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
W20260807 00:42:43.634106 22871 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:43.634169 22871 leader_election.cc:302] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cccd6eece80e43b89de54997299a5a12; no voters: 60bbc58eee394409ad018c34418ca4c3, 726e3fcb13a840b39434fd349f5f0558
I20260807 00:42:43.634335 23419 raft_consensus.cc:2726] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:42:44.161726 23425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:44.161421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:59277)
0807 00:42:44.161618 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:44.161668 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":96,"threads_started":1}
I20260807 00:42:44.258575 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:44.258230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:45173)
0807 00:42:44.258409 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:44.258479 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":68,"threads_started":1}
I20260807 00:42:44.402202 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:44.401920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:59539)
0807 00:42:44.402094 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:44.402140 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":118,"threads_started":1}
I20260807 00:42:45.162736 23430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:45.162409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:35087)
0807 00:42:45.162621 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:45.162672 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":74,"threads_started":1}
I20260807 00:42:45.259423 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:45.259149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:57627)
0807 00:42:45.259330 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:45.259385 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":85,"threads_started":1}
I20260807 00:42:45.403192 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:45.402849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:53925)
0807 00:42:45.403060 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:45.403133 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":89,"threads_started":1}
I20260807 00:42:45.436753 23434 raft_consensus.cc:482] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:45.436877 23434 raft_consensus.cc:504] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:45.437194 23434 leader_election.cc:288] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405), 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959)
W20260807 00:42:45.437690 22871 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
W20260807 00:42:45.437780 22872 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:45.437932 22872 leader_election.cc:302] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cccd6eece80e43b89de54997299a5a12; no voters: 60bbc58eee394409ad018c34418ca4c3, 726e3fcb13a840b39434fd349f5f0558
I20260807 00:42:45.438061 23434 raft_consensus.cc:2726] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:42:46.163511 23435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:46.163248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:49147)
0807 00:42:46.163417 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:46.163460 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260807 00:42:46.260311 23436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:46.260042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:40567)
0807 00:42:46.260215 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:46.260259 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260807 00:42:46.404248 23437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:46.403949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:34931)
0807 00:42:46.404140 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:46.404188 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":123,"threads_started":1}
I20260807 00:42:47.164435 23438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:47.164120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:47583)
0807 00:42:47.164319 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:47.164376 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":99,"threads_started":1}
I20260807 00:42:47.261200 23439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:47.260866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:45687)
0807 00:42:47.261089 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:47.261150 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
I20260807 00:42:47.404966 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:47.404698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:42367)
0807 00:42:47.404865 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:47.404910 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":85,"threads_started":1}
I20260807 00:42:47.632860 23442 raft_consensus.cc:482] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:47.633006 23442 raft_consensus.cc:504] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:47.633255 23442 leader_election.cc:288] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405), 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959)
W20260807 00:42:47.633736 22872 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
W20260807 00:42:47.633814 22871 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:47.633855 22871 leader_election.cc:302] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cccd6eece80e43b89de54997299a5a12; no voters: 60bbc58eee394409ad018c34418ca4c3, 726e3fcb13a840b39434fd349f5f0558
I20260807 00:42:47.633996 23442 raft_consensus.cc:2726] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:42:48.165411 23444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:48.165101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:49603)
0807 00:42:48.165311 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:48.165360 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":109,"threads_started":1}
I20260807 00:42:48.262110 23445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:48.261817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:56525)
0807 00:42:48.262012 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:48.262061 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":117,"threads_started":1}
I20260807 00:42:48.405810 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:48.405500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:36149)
0807 00:42:48.405692 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:48.405751 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260807 00:42:49.166312 23447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:49.166047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:47909)
0807 00:42:49.166200 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:49.166248 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":71,"threads_started":1}
I20260807 00:42:49.262899 23448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:49.262637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:37789)
0807 00:42:49.262797 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:49.262843 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260807 00:42:49.406857 23451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:49.406574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:55349)
0807 00:42:49.406757 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:49.406802 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":99,"threads_started":1}
I20260807 00:42:50.167098 23452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:50.166813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:49983)
0807 00:42:50.167003 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:50.167046 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260807 00:42:50.263603 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:50.263353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:46669)
0807 00:42:50.263517 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:50.263554 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":91,"threads_started":1}
I20260807 00:42:50.407625 23455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:50.407363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:53921)
0807 00:42:50.407511 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:50.407567 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260807 00:42:50.489140 23457 raft_consensus.cc:482] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:50.489289 23457 raft_consensus.cc:504] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:50.489547 23457 leader_election.cc:288] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405), 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959)
W20260807 00:42:50.490087 22872 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
W20260807 00:42:50.490226 22871 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:50.490310 22871 leader_election.cc:302] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cccd6eece80e43b89de54997299a5a12; no voters: 60bbc58eee394409ad018c34418ca4c3, 726e3fcb13a840b39434fd349f5f0558
I20260807 00:42:50.490439 23457 raft_consensus.cc:2726] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:42:51.167974 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:51.167699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:39997)
0807 00:42:51.167876 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:51.167923 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":90,"threads_started":1}
I20260807 00:42:51.264428 23459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:51.264140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:43643)
0807 00:42:51.264313 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:51.264373 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":78,"threads_started":1}
I20260807 00:42:51.408445 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:51.408158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:43153)
0807 00:42:51.408346 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:51.408393 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":107,"threads_started":1}
I20260807 00:42:52.168851 23462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:52.168577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:58011)
0807 00:42:52.168734 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:52.168787 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":70,"threads_started":1}
I20260807 00:42:52.265290 23465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:52.265000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:45485)
0807 00:42:52.265199 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:52.265241 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":107,"threads_started":1}
I20260807 00:42:52.409307 23466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:52.409030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:40975)
0807 00:42:52.409225 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:52.409263 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":94,"threads_started":1}
I20260807 00:42:53.169732 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:53.169375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:42293)
0807 00:42:53.169552 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:53.169648 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260807 00:42:53.266055 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:53.265793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:46659)
0807 00:42:53.265974 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:53.266021 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260807 00:42:53.410231 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:53.409887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:51235)
0807 00:42:53.410096 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:53.410150 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":101,"threads_started":1}
I20260807 00:42:54.170697 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:54.170322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:48233)
0807 00:42:54.170522 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:54.170644 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":112,"threads_started":1}
I20260807 00:42:54.266899 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:54.266630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:49867)
0807 00:42:54.266781 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:54.266838 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":67,"threads_started":1}
I20260807 00:42:54.411180 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:54.410831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:35593)
0807 00:42:54.411015 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:54.411109 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260807 00:42:54.580780 23476 raft_consensus.cc:482] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:54.580919 23476 raft_consensus.cc:504] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "60bbc58eee394409ad018c34418ca4c3" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 40405 } } peers { permanent_uuid: "726e3fcb13a840b39434fd349f5f0558" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 41959 } } peers { permanent_uuid: "cccd6eece80e43b89de54997299a5a12" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 33775 } }
I20260807 00:42:54.581216 23476 leader_election.cc:288] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405), 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959)
W20260807 00:42:54.581782 22871 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 726e3fcb13a840b39434fd349f5f0558 (127.1.116.130:41959): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
W20260807 00:42:54.581923 22872 leader_election.cc:341] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 60bbc58eee394409ad018c34418ca4c3 (127.1.116.131:40405): Not found: Tablet not found: 10dc49eea9e647fc8bc6ba286c5429db
I20260807 00:42:54.581969 22872 leader_election.cc:302] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cccd6eece80e43b89de54997299a5a12; no voters: 60bbc58eee394409ad018c34418ca4c3, 726e3fcb13a840b39434fd349f5f0558
I20260807 00:42:54.582090 23476 raft_consensus.cc:2726] T 10dc49eea9e647fc8bc6ba286c5429db P cccd6eece80e43b89de54997299a5a12 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:42:55.171471 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:55.171198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:47889)
0807 00:42:55.171353 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:55.171410 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260807 00:42:55.267717 23478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:55.267447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:47609)
0807 00:42:55.267605 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:55.267659 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":68,"threads_started":1}
I20260807 00:42:55.412004 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:55.411722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:35369)
0807 00:42:55.411909 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:55.411953 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260807 00:42:56.172324 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:56.172044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:45519)
0807 00:42:56.172228 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:56.172270 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":93,"threads_started":1}
I20260807 00:42:56.268560 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:56.268312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:33895)
0807 00:42:56.268488 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:56.268526 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":80,"threads_started":1}
I20260807 00:42:56.412880 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:56.412593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:37089)
0807 00:42:56.412785 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:56.412842 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":106,"threads_started":1}
I20260807 00:42:57.173317 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:57.173036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.129:33183)
0807 00:42:57.173211 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:57.173257 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":72,"threads_started":1}
I20260807 00:42:57.269626 23487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:57.269196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.130:47323)
0807 00:42:57.269453 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:57.269527 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":95,"threads_started":1}
I20260807 00:42:57.413760 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:57.413502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36905 (local address 127.1.116.131:53201)
0807 00:42:57.413679 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:57.413717 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":92,"threads_started":1}
I20260807 00:42:57.418224 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 22806
I20260807 00:42:57.434207 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 23037
I20260807 00:42:57.444351 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 23223
2026-08-07T00:42:57Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18914 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:42:57 dist-test-slave-h6n0 krb5kdc[23492](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:42:57 dist-test-slave-h6n0 krb5kdc[23492](info): set up 2 sockets
Aug 07 00:42:57 dist-test-slave-h6n0 krb5kdc[23492](info): commencing operation
krb5kdc: starting...
W20260807 00:42:58.706403 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:42:58 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:42:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:42:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:42:58.841358 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36907
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:58.936421 23508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:58.936661 23508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:58.936685 23508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:58.937887 23508 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 00:42:58.937937 23508 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 00:42:58.937951 23508 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 00:42:58.937963 23508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:42:58.937976 23508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:58.937989 23508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:58.938004 23508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:42:58.938016 23508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:42:58.939667 23508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:58.939970 23508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:58.940251 23508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:58.946169 23519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:58.946230 23521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:58.946211 23518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:58.946220 23508 server_base.cc:1034] running on GCE node
I20260807 00:42:58.946741 23508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:58.947150 23508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:58.948309 23508 hybrid_clock.cc:648] HybridClock initialized: now 1786063378948275 us; error 56 us; skew 500 ppm
Aug 07 00:42:58 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063378, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:58.951318 23508 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:42:58.951869 23508 webserver.cc:460] Webserver started at http://127.1.116.190:40267/ using document root <none> and password file <none>
I20260807 00:42:58.952162 23508 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:58.952219 23508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:58.952389 23508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:58.953581 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "8c8d0e67e9a941ed973e021b09adc455"
format_stamp: "Formatted at 2026-08-07 00:42:58 on dist-test-slave-h6n0"
server_key: "91568299d997094718f16a5f81ad035e"
server_key_iv: "348a8fb47a2e6f6ab774fcd2075ccd5e"
server_key_version: "encryptionkey@0"
I20260807 00:42:58.953996 23508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "8c8d0e67e9a941ed973e021b09adc455"
format_stamp: "Formatted at 2026-08-07 00:42:58 on dist-test-slave-h6n0"
server_key: "91568299d997094718f16a5f81ad035e"
server_key_iv: "348a8fb47a2e6f6ab774fcd2075ccd5e"
server_key_version: "encryptionkey@0"
I20260807 00:42:58.955628 23508 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:42:58.956462 23529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:58.956672 23508 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:58.956794 23508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "8c8d0e67e9a941ed973e021b09adc455"
format_stamp: "Formatted at 2026-08-07 00:42:58 on dist-test-slave-h6n0"
server_key: "91568299d997094718f16a5f81ad035e"
server_key_iv: "348a8fb47a2e6f6ab774fcd2075ccd5e"
server_key_version: "encryptionkey@0"
I20260807 00:42:58.956874 23508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:58.975519 23508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:58.977702 23508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:58.977882 23508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:58.982999 23508 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:36907
I20260807 00:42:58.983047 23599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:36907 every 8 connection(s)
I20260807 00:42:58.983479 23508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:42:58.984459 23602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:58.986909 23602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Bootstrap starting.
I20260807 00:42:58.987468 23602 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:58.987722 23602 log.cc:824] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:58.988162 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 23508
I20260807 00:42:58.988294 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:42:58.988548 1490 external_mini_cluster.cc:1428] Setting key bb7ca8b3f3bd236d32db4075ab872974
I20260807 00:42:58.988590 23602 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: No bootstrap required, opened a new log
I20260807 00:42:58.991237 23602 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:42:58.991376 23602 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:58.991398 23602 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8d0e67e9a941ed973e021b09adc455, State: Initialized, Role: FOLLOWER
I20260807 00:42:58.991533 23602 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:42:58.991616 23602 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:42:58.991639 23602 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:42:58.991715 23602 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:58.992280 23602 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:42:58.992370 23602 leader_election.cc:302] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8d0e67e9a941ed973e021b09adc455; no voters:
I20260807 00:42:58.992604 23602 leader_election.cc:288] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:42:58.992645 23610 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:58.992767 23610 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 LEADER]: Becoming Leader. State: Replica: 8c8d0e67e9a941ed973e021b09adc455, State: Running, Role: LEADER
I20260807 00:42:58.992898 23602 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:42:58.992977 23610 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:42:58.993448 23610 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c8d0e67e9a941ed973e021b09adc455. Latest consensus state: current_term: 1 leader_uuid: "8c8d0e67e9a941ed973e021b09adc455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } } }
I20260807 00:42:58.993607 23610 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:58.993884 23621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:42:58.993860 23613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c8d0e67e9a941ed973e021b09adc455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } } }
I20260807 00:42:58.994041 23613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: This master's current role is: LEADER
I20260807 00:42:58.994551 23621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:42:58.996452 23621 catalog_manager.cc:1347] Generated new cluster ID: 69a69617da2e4f3e83e1fc31938d609e
I20260807 00:42:58.996510 23621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:42:58 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063378, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:59.002005 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:58.989156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55208 (local address 127.1.116.190:36907)
0807 00:42:58.989398 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:58.989408 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:58.989438 (+ 30us) server_negotiation.cc:407] Connection header received
0807 00:42:58.990053 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:58.990061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:58.990113 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:58.990247 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:58.991134 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:58.992781 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:58.993957 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:58.994654 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:58.994786 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:58.997799 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:58.997804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:58.997805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:58.997817 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:59.000023 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.000502 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.000505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.000506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.000619 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.000868 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.000869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.000869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.001187 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:59.001340 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.001495 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.001786 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":147,"threads_started":1}
I20260807 00:42:59.013671 23621 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:42:59.015089 23621 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:42:59.018808 23621 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Generated new TSK 0
I20260807 00:42:59.018980 23621 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:42:59.051112 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36907
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:59.142915 23643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:59.143148 23643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:59.143173 23643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:59.144358 23643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:59.144404 23643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:59.144431 23643 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:42:59.145907 23643 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:59.146189 23643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:59.146466 23643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:59.152115 23651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:59.152184 23649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:59.152222 23643 server_base.cc:1034] running on GCE node
W20260807 00:42:59.152113 23654 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:59.152702 23643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:59.152994 23643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:59.154182 23643 hybrid_clock.cc:648] HybridClock initialized: now 1786063379154169 us; error 30 us; skew 500 ppm
Aug 07 00:42:59 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063379, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:59.157097 23643 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:42:59.157610 23643 webserver.cc:460] Webserver started at http://127.1.116.129:43291/ using document root <none> and password file <none>
I20260807 00:42:59.157856 23643 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:59.157927 23643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:59.158087 23643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:59.159333 23643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "409276227a7a4bb09bc09c5e364c48dd"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "6c70570bde0bf987e47ed39327d96661"
server_key_iv: "38200b2bbd83061dd902cd25f02f4c36"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.159724 23643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "409276227a7a4bb09bc09c5e364c48dd"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "6c70570bde0bf987e47ed39327d96661"
server_key_iv: "38200b2bbd83061dd902cd25f02f4c36"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.161396 23643 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:42:59.162420 23664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.162698 23643 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:59.162817 23643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "409276227a7a4bb09bc09c5e364c48dd"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "6c70570bde0bf987e47ed39327d96661"
server_key_iv: "38200b2bbd83061dd902cd25f02f4c36"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.162906 23643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:59.175283 23643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:59.177279 23643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:59.177456 23643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:59.177727 23643 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:59.178087 23643 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:59.178146 23643 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.178184 23643 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:59.178241 23643 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.185662 23643 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:35689
I20260807 00:42:59.185738 23812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:35689 every 8 connection(s)
I20260807 00:42:59.186076 23643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:42:59.187595 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 23643
I20260807 00:42:59.187721 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:42:59.187937 1490 external_mini_cluster.cc:1428] Setting key 465a7d21f421d3adce54f9b90df34c4b
Aug 07 00:42:59 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063379, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:59.196843 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.186895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:51171 (local address 127.1.116.190:36907)
0807 00:42:59.186967 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.186970 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.187200 (+ 230us) server_negotiation.cc:407] Connection header received
0807 00:42:59.187835 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.187836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.187850 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.187915 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:59.189142 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.189695 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.190841 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.191060 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.191146 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.193578 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:59.193579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:59.193580 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:59.193588 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:59.195344 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.195774 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.195775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.195776 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.195837 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.196103 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.196104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.196104 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.196229 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:59.196316 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.196636 (+ 320us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.196736 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:42:59.197059 23816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.186853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:51171)
0807 00:42:59.187053 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.187097 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.187709 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.187991 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.187996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.188799 (+ 803us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:59.189023 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.189029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.189800 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.189801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.190741 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.190744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.191158 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.191159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.191295 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.191853 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:59.191862 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:59.193480 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:59.195434 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:59.195441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:59.195444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:59.195675 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:59.195906 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:59.195907 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:59.195907 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:59.196025 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:59.196305 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:59.196313 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:59.196462 (+ 149us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.196577 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.196789 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:42:59.197636 23813 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36907
I20260807 00:42:59.197762 23813 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:59.197970 23813 heartbeater.cc:511] Master 127.1.116.190:36907 requested a full tablet report, sending...
I20260807 00:42:59.198710 23556 ts_manager.cc:194] Registered new tserver with Master: 409276227a7a4bb09bc09c5e364c48dd (127.1.116.129:35689)
I20260807 00:42:59.199417 23556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:51171
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:42:59.214665 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36907
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:59.311117 23819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:59.311523 23819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:59.311600 23819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:59.314213 23819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:59.314353 23819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:59.314428 23819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:42:59.317186 23819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23819
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:59.318819 23819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:59.320618 23819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:59.326761 23824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:59.326918 23825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:59.326749 23829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:59.327221 23819 server_base.cc:1034] running on GCE node
I20260807 00:42:59.327479 23819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:59.327852 23819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:59.328998 23819 hybrid_clock.cc:648] HybridClock initialized: now 1786063379328966 us; error 57 us; skew 500 ppm
Aug 07 00:42:59 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063379, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:59.332404 23819 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:42:59.332880 23819 webserver.cc:460] Webserver started at http://127.1.116.130:37209/ using document root <none> and password file <none>
I20260807 00:42:59.333137 23819 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:59.333207 23819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:59.333374 23819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:59.334812 23819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "2f54105b7a8641028afc0b021ba22361"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "b50ca90d0a3795ee71d18ee090caf4de"
server_key_iv: "2ffbf4052a28f561dd4b70dc46c7d21b"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.335254 23819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "2f54105b7a8641028afc0b021ba22361"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "b50ca90d0a3795ee71d18ee090caf4de"
server_key_iv: "2ffbf4052a28f561dd4b70dc46c7d21b"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.337311 23819 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260807 00:42:59.338479 23838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.338774 23819 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:42:59.338886 23819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "2f54105b7a8641028afc0b021ba22361"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "b50ca90d0a3795ee71d18ee090caf4de"
server_key_iv: "2ffbf4052a28f561dd4b70dc46c7d21b"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.339004 23819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:59.354067 23819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:59.357163 23819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:59.357342 23819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:59.357595 23819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:59.358034 23819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:59.358088 23819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.358136 23819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:59.358162 23819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.367350 23819 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:45141
I20260807 00:42:59.367403 23984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:45141 every 8 connection(s)
I20260807 00:42:59.367791 23819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:42:59.371325 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 23819
I20260807 00:42:59.371471 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:42:59.371717 1490 external_mini_cluster.cc:1428] Setting key 9f268327201dbfc45bfba4cabae0def4
Aug 07 00:42:59 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063379, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:59.379386 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.368705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:36683 (local address 127.1.116.190:36907)
0807 00:42:59.368770 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.368774 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.369059 (+ 285us) server_negotiation.cc:407] Connection header received
0807 00:42:59.369620 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.369621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.369636 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.369695 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:59.371211 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.371981 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.373577 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.373783 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.373844 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.376235 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:59.376236 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:59.376236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:59.376245 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:59.377782 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.378248 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.378250 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.378251 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.378321 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.378644 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.378645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.378646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.378785 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:59.378868 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.379178 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.379277 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 00:42:59.379812 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.368696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:36683)
0807 00:42:59.368893 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.368962 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.369498 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.369845 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.369851 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.370766 (+ 915us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:59.371101 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.371107 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.372173 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.372175 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.373479 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.373483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.373880 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.373881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.374035 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.374751 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:59.374761 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:59.376138 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:59.377885 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:59.377896 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:59.377898 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:59.378166 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:59.378393 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:59.378394 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:59.378395 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:59.378522 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:59.378859 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:59.378867 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:59.379030 (+ 163us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.379142 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.379350 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":88,"threads_started":1}
I20260807 00:42:59.380414 23985 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36907
I20260807 00:42:59.380528 23985 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:59.380738 23985 heartbeater.cc:511] Master 127.1.116.190:36907 requested a full tablet report, sending...
I20260807 00:42:59.381186 23556 ts_manager.cc:194] Registered new tserver with Master: 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141)
I20260807 00:42:59.381606 23556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:36683
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:42:59.401114 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36907
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:42:59.488441 23991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:42:59.488649 23991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:42:59.488673 23991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:42:59.489727 23991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:42:59.489765 23991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:42:59.489790 23991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:42:59.491236 23991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:42:59.491425 23991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:42:59.491598 23991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:42:59.497028 23998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:59.497048 24003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:42:59.497063 24000 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:42:59.497571 23991 server_base.cc:1034] running on GCE node
I20260807 00:42:59.497753 23991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:42:59.497944 23991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:42:59.499112 23991 hybrid_clock.cc:648] HybridClock initialized: now 1786063379499098 us; error 41 us; skew 500 ppm
Aug 07 00:42:59 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063379, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:42:59.501896 23991 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:42:59.502372 23991 webserver.cc:460] Webserver started at http://127.1.116.131:34283/ using document root <none> and password file <none>
I20260807 00:42:59.502657 23991 fs_manager.cc:359] Metadata directory not provided
I20260807 00:42:59.502727 23991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:42:59.502869 23991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:42:59.503865 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "74235eae63044f5b955ed438796ab92b"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "f5950fbc4efb3657f58ef4c55d8d7599"
server_key_iv: "da083f10e860f692867731b1b62fb84a"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.504232 23991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "74235eae63044f5b955ed438796ab92b"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "f5950fbc4efb3657f58ef4c55d8d7599"
server_key_iv: "da083f10e860f692867731b1b62fb84a"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.505856 23991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:42:59.506769 24014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.507108 23991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:42:59.507253 23991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "74235eae63044f5b955ed438796ab92b"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "f5950fbc4efb3657f58ef4c55d8d7599"
server_key_iv: "da083f10e860f692867731b1b62fb84a"
server_key_version: "encryptionkey@0"
I20260807 00:42:59.507357 23991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:42:59.522641 23991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:42:59.524659 23991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:42:59.524833 23991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:42:59.525069 23991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:42:59.525390 23991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:42:59.525437 23991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.525460 23991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:42:59.525498 23991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:42:59.532584 23991 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44575
I20260807 00:42:59.532636 24165 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44575 every 8 connection(s)
I20260807 00:42:59.532927 23991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:42:59.537948 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 23991
I20260807 00:42:59.538050 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:42:59.538249 1490 external_mini_cluster.cc:1428] Setting key dfbf259664d11c7ddfa4deef77a75fb3
Aug 07 00:42:59 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063379, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:42:59.543892 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.533772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:52511 (local address 127.1.116.190:36907)
0807 00:42:59.533835 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.533838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.534130 (+ 292us) server_negotiation.cc:407] Connection header received
0807 00:42:59.534734 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.534735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.534747 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.534825 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:59.535889 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.536629 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.537870 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.538197 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.538287 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.540904 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:59.540905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:59.540905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:59.540911 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:59.542647 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.543046 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.543048 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.543048 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.543108 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.543360 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.543361 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.543361 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.543468 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:59.543526 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.543762 (+ 236us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.543827 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:42:59.544076 24169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.533726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:52511)
0807 00:42:59.533941 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.533988 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.534610 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.534910 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.534914 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.535542 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:42:59.535784 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.535789 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.536809 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.536810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.537754 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.537757 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.538300 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.538302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.538468 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.539230 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:42:59.539242 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:42:59.540816 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:42:59.542726 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:59.542732 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:59.542734 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:59.542948 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:59.543188 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:42:59.543189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:42:59.543189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:42:59.543310 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:42:59.543517 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:42:59.543520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:42:59.543640 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.543726 (+ 86us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.543855 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":124,"threads_started":1}
I20260807 00:42:59.544644 24167 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36907
I20260807 00:42:59.544757 24167 heartbeater.cc:464] Registering TS with master...
I20260807 00:42:59.544979 24167 heartbeater.cc:511] Master 127.1.116.190:36907 requested a full tablet report, sending...
I20260807 00:42:59.545462 23556 ts_manager.cc:194] Registered new tserver with Master: 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
I20260807 00:42:59.545892 23556 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:52511
I20260807 00:42:59.550609 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:42:59.558370 23608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.552069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55224 (local address 127.1.116.190:36907)
0807 00:42:59.552154 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.552156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.552243 (+ 87us) server_negotiation.cc:407] Connection header received
0807 00:42:59.552306 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.552307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.552316 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.552369 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:42:59.552923 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.553418 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.554116 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.554301 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.554355 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.555577 (+ 1222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:42:59.555578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:42:59.555579 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:42:59.555584 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:42:59.557041 (+ 1457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.557422 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.557422 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.557423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.557475 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:42:59.557706 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:42:59.557707 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:42:59.557707 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:42:59.557868 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:42:59.557954 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.558167 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.558238 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:42:59.560542 23556 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:42:59.562489 23556 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:42:59.570073 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.565475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44575 (local address 127.0.0.1:33084)
0807 00:42:59.565790 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.565800 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.565851 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.566315 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.566318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.566323 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:59.566427 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.566430 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.568023 (+ 1593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.568025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.568994 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.568997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.569705 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.569706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.569809 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.569824 (+ 15us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.569929 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.569994 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":28,"thread_start_us":129,"threads_started":1}
I20260807 00:42:59.570086 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.565735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.1.116.131:44575)
0807 00:42:59.566053 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.566056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.566068 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:42:59.566110 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.566112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.566129 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.566218 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:59.566577 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.567806 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.569094 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.569610 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.569703 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.569769 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.569948 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.569994 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":100,"threads_started":1}
I20260807 00:42:59.570385 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.565575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43958 (local address 127.1.116.129:35689)
0807 00:42:59.566590 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.566594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.566604 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:42:59.566647 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.566649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.566660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.566755 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:59.566864 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.567553 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.569149 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.569883 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.570164 (+ 281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.570246 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.570272 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.570306 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":982,"thread_start_us":99,"threads_started":1}
I20260807 00:42:59.570765 24189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.565508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:45141 (local address 127.0.0.1:37636)
0807 00:42:59.565963 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.565971 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.566015 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.566315 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.566318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.566321 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:59.566427 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.566431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.567895 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.567896 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.569042 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.569045 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.570001 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.570002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.570072 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.570081 (+ 9us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.570113 (+ 32us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.570673 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":39,"thread_start_us":113,"threads_started":1}
I20260807 00:42:59.570839 24188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.565450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35689 (local address 127.0.0.1:43958)
0807 00:42:59.565664 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.565710 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.565818 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.566756 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.566757 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.566760 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:59.566801 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.566803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.567665 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.567666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.569043 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.569045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.569996 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.569998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.570072 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.570079 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.570113 (+ 34us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.570693 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":512,"thread_start_us":52,"threads_started":1}
I20260807 00:42:59.570992 24080 tablet_service.cc:1467] Processing CreateTablet for tablet 001b64164212429eb1aeb6378da490a6 (DEFAULT_TABLE table=test-table [id=3c967164d35b47f09561168df6b97032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:59.571219 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.565826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37636 (local address 127.1.116.130:45141)
0807 00:42:59.566049 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.566054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.566068 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:42:59.566111 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.566113 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.566129 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.566218 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:59.566534 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.567764 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.569153 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.569883 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.571001 (+ 1118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.571065 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.571090 (+ 25us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.571135 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":90,"threads_started":1}
I20260807 00:42:59.571426 24080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001b64164212429eb1aeb6378da490a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:59.571842 23903 tablet_service.cc:1467] Processing CreateTablet for tablet 001b64164212429eb1aeb6378da490a6 (DEFAULT_TABLE table=test-table [id=3c967164d35b47f09561168df6b97032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:59.572191 23903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001b64164212429eb1aeb6378da490a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:59.574720 24193 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Bootstrap starting.
I20260807 00:42:59.575335 23728 tablet_service.cc:1467] Processing CreateTablet for tablet 001b64164212429eb1aeb6378da490a6 (DEFAULT_TABLE table=test-table [id=3c967164d35b47f09561168df6b97032]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:42:59.575692 23728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 001b64164212429eb1aeb6378da490a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:42:59.575701 24193 tablet_bootstrap.cc:654] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:59.576460 24193 log.cc:824] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:59.576529 24195 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Bootstrap starting.
I20260807 00:42:59.577371 24195 tablet_bootstrap.cc:654] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:59.577731 24193 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: No bootstrap required, opened a new log
I20260807 00:42:59.577834 24193 ts_tablet_manager.cc:1397] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:42:59.577997 24195 log.cc:824] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:59.578948 24195 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: No bootstrap required, opened a new log
I20260807 00:42:59.579032 24195 ts_tablet_manager.cc:1397] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:42:59.579417 24198 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap starting.
I20260807 00:42:59.580075 24193 raft_consensus.cc:348] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.580281 24193 raft_consensus.cc:374] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:59.580327 24193 raft_consensus.cc:729] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74235eae63044f5b955ed438796ab92b, State: Initialized, Role: FOLLOWER
I20260807 00:42:59.580246 24198 tablet_bootstrap.cc:654] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Neither blocks nor log segments found. Creating new log.
I20260807 00:42:59.580454 24193 consensus_queue.cc:260] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.580808 24193 ts_tablet_manager.cc:1428] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:42:59.580860 24167 heartbeater.cc:503] Master 127.1.116.190:36907 was elected leader, sending a full tablet report...
I20260807 00:42:59.580655 24195 raft_consensus.cc:348] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.581167 24195 raft_consensus.cc:374] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:59.581250 24195 raft_consensus.cc:729] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f54105b7a8641028afc0b021ba22361, State: Initialized, Role: FOLLOWER
I20260807 00:42:59.581377 24198 log.cc:824] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Log is configured to *not* fsync() on all Append() calls
I20260807 00:42:59.581418 24195 consensus_queue.cc:260] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.581750 24195 ts_tablet_manager.cc:1428] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:42:59.582058 23985 heartbeater.cc:503] Master 127.1.116.190:36907 was elected leader, sending a full tablet report...
I20260807 00:42:59.582160 24198 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: No bootstrap required, opened a new log
I20260807 00:42:59.582247 24198 ts_tablet_manager.cc:1397] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:42:59.584240 24198 raft_consensus.cc:348] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.584412 24198 raft_consensus.cc:374] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:42:59.584462 24198 raft_consensus.cc:729] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409276227a7a4bb09bc09c5e364c48dd, State: Initialized, Role: FOLLOWER
I20260807 00:42:59.584578 24198 consensus_queue.cc:260] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.584852 24198 ts_tablet_manager.cc:1428] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:42:59.584954 23813 heartbeater.cc:503] Master 127.1.116.190:36907 was elected leader, sending a full tablet report...
W20260807 00:42:59.618480 23987 tablet.cc:2369] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:42:59.686887 23814 tablet.cc:2369] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:42:59.742273 24205 raft_consensus.cc:482] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:42:59.742416 24205 raft_consensus.cc:504] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.742904 24205 leader_election.cc:288] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575), 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141)
I20260807 00:42:59.747644 24207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.743218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44575 (local address 127.1.116.129:42673)
0807 00:42:59.743406 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.743421 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.743510 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.743705 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.743706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.743709 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:59.743859 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.743862 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.744830 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.744835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.746351 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.746355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.747225 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.747227 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.747335 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.747347 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.747409 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.747502 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":113,"thread_start_us":85,"threads_started":1}
I20260807 00:42:59.747618 24192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.743328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:40043 (local address 127.1.116.130:45141)
0807 00:42:59.743386 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.743389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.743522 (+ 133us) server_negotiation.cc:407] Connection header received
0807 00:42:59.743570 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.743571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.743583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.743661 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:59.743955 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.744713 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.746466 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.747094 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.747165 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.747174 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.747451 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.747524 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 00:42:59.747618 24191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.743337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:42673 (local address 127.1.116.131:44575)
0807 00:42:59.743401 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.743403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.743517 (+ 114us) server_negotiation.cc:407] Connection header received
0807 00:42:59.743597 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.743598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.743607 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.743667 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:42:59.743954 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.744723 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.746466 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.747086 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.747165 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.747174 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.747449 (+ 275us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.747524 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:42:59.747648 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.743218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:45141 (local address 127.1.116.129:40043)
0807 00:42:59.743406 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.743421 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:42:59.743510 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:42:59.743705 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:42:59.743706 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:42:59.743710 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:42:59.743862 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.743864 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.744830 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.744831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.746351 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:42:59.746355 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.747227 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.747228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:42:59.747335 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.747345 (+ 10us) client_negotiation.cc:769] Sending connection context
0807 00:42:59.747410 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 00:42:59.747502 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"mutex_wait_us":105,"thread_start_us":96,"threads_started":1}
I20260807 00:42:59.748631 23926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001b64164212429eb1aeb6378da490a6" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f54105b7a8641028afc0b021ba22361" is_pre_election: true
I20260807 00:42:59.748806 23926 raft_consensus.cc:2454] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 0.
I20260807 00:42:59.748749 24102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001b64164212429eb1aeb6378da490a6" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74235eae63044f5b955ed438796ab92b" is_pre_election: true
I20260807 00:42:59.748950 24102 raft_consensus.cc:2454] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 0.
I20260807 00:42:59.749104 23669 leader_election.cc:302] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f54105b7a8641028afc0b021ba22361, 409276227a7a4bb09bc09c5e364c48dd; no voters:
I20260807 00:42:59.749284 24205 raft_consensus.cc:2781] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:42:59.749377 24205 raft_consensus.cc:482] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:42:59.749414 24205 raft_consensus.cc:3037] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:59.750330 24205 raft_consensus.cc:504] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.750516 24205 leader_election.cc:288] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 election: Requested vote from peers 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575), 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141)
I20260807 00:42:59.750691 24102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001b64164212429eb1aeb6378da490a6" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74235eae63044f5b955ed438796ab92b"
I20260807 00:42:59.750728 23926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "001b64164212429eb1aeb6378da490a6" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f54105b7a8641028afc0b021ba22361"
I20260807 00:42:59.750787 24102 raft_consensus.cc:3037] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:59.750806 23926 raft_consensus.cc:3037] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:42:59.751875 23926 raft_consensus.cc:2454] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 1.
I20260807 00:42:59.751871 24102 raft_consensus.cc:2454] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 1.
I20260807 00:42:59.752099 23669 leader_election.cc:302] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f54105b7a8641028afc0b021ba22361, 409276227a7a4bb09bc09c5e364c48dd; no voters:
I20260807 00:42:59.752210 24205 raft_consensus.cc:2781] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:42:59.752424 24205 raft_consensus.cc:686] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 1 LEADER]: Becoming Leader. State: Replica: 409276227a7a4bb09bc09c5e364c48dd, State: Running, Role: LEADER
I20260807 00:42:59.752555 24205 consensus_queue.cc:237] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:42:59.753783 23555 catalog_manager.cc:5495] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 409276227a7a4bb09bc09c5e364c48dd (127.1.116.129). New cstate: current_term: 1 leader_uuid: "409276227a7a4bb09bc09c5e364c48dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } health_report { overall_health: UNKNOWN } } }
W20260807 00:42:59.783525 24168 tablet.cc:2369] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:42:59.798321 24190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.795632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.1.116.129:35689)
0807 00:42:59.795920 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 00:42:59.795922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:42:59.795930 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:42:59.795971 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:42:59.795972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:42:59.795994 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:42:59.796085 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:42:59.796252 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.796739 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.797500 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:42:59.797692 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:42:59.797768 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:42:59.797951 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:42:59.798077 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:42:59.798176 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 00:42:59.798200 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 00:42:59.798221 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255}
I20260807 00:42:59.800544 23926 raft_consensus.cc:1261] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Refusing update from remote peer 409276227a7a4bb09bc09c5e364c48dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:59.800544 24102 raft_consensus.cc:1261] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Refusing update from remote peer 409276227a7a4bb09bc09c5e364c48dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:42:59.801012 24209 consensus_queue.cc:1035] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:59.801174 24205 consensus_queue.cc:1035] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:42:59.804934 24214 mvcc.cc:205] Tried to move back new op lower bound from 7315715603659415552 to 7315715603467386880. Current Snapshot: MvccSnapshot[applied={T|T < 7315715603659415552}]
I20260807 00:42:59.805519 24213 mvcc.cc:205] Tried to move back new op lower bound from 7315715603659415552 to 7315715603467386880. Current Snapshot: MvccSnapshot[applied={T|T < 7315715603659415552}]
I20260807 00:42:59.807224 23555 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:59.807353 23555 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:42:59.808331 23555 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Sending DeleteTablet for 3 replicas of tablet 001b64164212429eb1aeb6378da490a6
I20260807 00:42:59.808710 23728 tablet_service.cc:1514] Processing DeleteTablet for tablet 001b64164212429eb1aeb6378da490a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:59 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:43958
I20260807 00:42:59.808802 23903 tablet_service.cc:1514] Processing DeleteTablet for tablet 001b64164212429eb1aeb6378da490a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:59 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37636
I20260807 00:42:59.809005 24080 tablet_service.cc:1514] Processing DeleteTablet for tablet 001b64164212429eb1aeb6378da490a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:59 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:33084
I20260807 00:42:59.809072 24227 tablet_replica.cc:330] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: stopping tablet replica
I20260807 00:42:59.809185 24227 raft_consensus.cc:2229] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:42:59.809290 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 23643
I20260807 00:42:59.809303 24227 raft_consensus.cc:2258] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:59.809815 24227 ts_tablet_manager.cc:1905] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:59.815168 24227 ts_tablet_manager.cc:1918] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:59.815258 24227 log.cc:1196] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/001b64164212429eb1aeb6378da490a6
I20260807 00:42:59.815580 24227 ts_tablet_manager.cc:1939] T 001b64164212429eb1aeb6378da490a6 P 2f54105b7a8641028afc0b021ba22361: Deleting consensus metadata
I20260807 00:42:59.816116 23535 catalog_manager.cc:4841] TS 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141): tablet 001b64164212429eb1aeb6378da490a6 (table test-table [id=3c967164d35b47f09561168df6b97032]) successfully deleted
I20260807 00:42:59.821818 24228 tablet_replica.cc:330] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: stopping tablet replica
I20260807 00:42:59.821964 24228 raft_consensus.cc:2229] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:42:59.822096 24228 raft_consensus.cc:2258] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:42:59.823407 24228 ts_tablet_manager.cc:1905] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:42:59.824959 24228 ts_tablet_manager.cc:1918] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:42:59.825094 24228 log.cc:1196] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/001b64164212429eb1aeb6378da490a6
I20260807 00:42:59.825450 24228 ts_tablet_manager.cc:1939] T 001b64164212429eb1aeb6378da490a6 P 74235eae63044f5b955ed438796ab92b: Deleting consensus metadata
I20260807 00:42:59.826043 23531 catalog_manager.cc:4841] TS 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575): tablet 001b64164212429eb1aeb6378da490a6 (table test-table [id=3c967164d35b47f09561168df6b97032]) successfully deleted
W20260807 00:42:59.830765 23530 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:35689 (error 108)
I20260807 00:42:59.831382 24187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:42:59.831238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35689 (local address 127.0.0.1:43982)
0807 00:42:59.831310 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 00:42:59.831352 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 00:42:59.831532 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 23819
W20260807 00:42:59.831557 23530 catalog_manager.cc:4584] TS 409276227a7a4bb09bc09c5e364c48dd (127.1.116.129:35689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 001b64164212429eb1aeb6378da490a6: Network error: Client connection negotiation failed: client connection to 127.1.116.129:35689: connect: Connection refused (error 111)
I20260807 00:42:59.842053 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 23991
I20260807 00:42:59.852416 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 23508
I20260807 00:43:19.865897 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36907
--webserver_interface=127.1.116.190
--webserver_port=40267
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:19.951838 24231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:19.952052 24231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:19.952090 24231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:19.953195 24231 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 00:43:19.953256 24231 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 00:43:19.953280 24231 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 00:43:19.953299 24231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:43:19.953339 24231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:19.953368 24231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:19.953399 24231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:43:19.953426 24231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:43:19.954933 24231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=40267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:19.955206 24231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:19.955443 24231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:43:19.966094 24231 server_base.cc:1034] running on GCE node
W20260807 00:43:19.966113 24240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:19.966125 24237 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:19.966090 24238 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:19.966950 24231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:19.967404 24231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:19.968560 24231 hybrid_clock.cc:648] HybridClock initialized: now 1786063399968546 us; error 59 us; skew 500 ppm
Aug 07 00:43:19 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063399, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:19.971684 24231 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:43:19.972141 24231 webserver.cc:460] Webserver started at http://127.1.116.190:40267/ using document root <none> and password file <none>
I20260807 00:43:19.972440 24231 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:19.972507 24231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:19.974120 24231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:43:19.975091 24248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:19.975366 24231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:43:19.975488 24231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "8c8d0e67e9a941ed973e021b09adc455"
format_stamp: "Formatted at 2026-08-07 00:42:58 on dist-test-slave-h6n0"
server_key: "91568299d997094718f16a5f81ad035e"
server_key_iv: "348a8fb47a2e6f6ab774fcd2075ccd5e"
server_key_version: "encryptionkey@0"
I20260807 00:43:19.975962 24231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:19.993322 24231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:19.995466 24231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:19.995638 24231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:20.000941 24231 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:36907
I20260807 00:43:20.000993 24326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:36907 every 8 connection(s)
I20260807 00:43:20.001446 24231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:43:20.003185 24328 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:43:20.003379 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 24231
I20260807 00:43:20.003723 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:35689
--local_ip_for_outbound_sockets=127.1.116.129
--tserver_master_addrs=127.1.116.190:36907
--webserver_port=43291
--webserver_interface=127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:43:20.004030 24328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Bootstrap starting.
I20260807 00:43:20.015112 24328 log.cc:824] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:20.017983 24328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:43:20.018347 24328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Bootstrap complete.
I20260807 00:43:20.021652 24328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:20.021960 24328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8d0e67e9a941ed973e021b09adc455, State: Initialized, Role: FOLLOWER
I20260807 00:43:20.022117 24328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:20.022189 24328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:43:20.022230 24328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:43:20.022281 24328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:43:20.023591 24328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:20.023705 24328 leader_election.cc:302] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8d0e67e9a941ed973e021b09adc455; no voters:
I20260807 00:43:20.023864 24328 leader_election.cc:288] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:43:20.023947 24333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:43:20.024125 24328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:43:20.024250 24333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 LEADER]: Becoming Leader. State: Replica: 8c8d0e67e9a941ed973e021b09adc455, State: Running, Role: LEADER
I20260807 00:43:20.024385 24333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:20.024751 24335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8c8d0e67e9a941ed973e021b09adc455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } } }
I20260807 00:43:20.024848 24335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:20.025113 24336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c8d0e67e9a941ed973e021b09adc455. Latest consensus state: current_term: 2 leader_uuid: "8c8d0e67e9a941ed973e021b09adc455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } } }
I20260807 00:43:20.025240 24336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:20.025349 24345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:43:20.026503 24345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:43:20.026643 24345 catalog_manager.cc:1259] Loaded cluster ID: 69a69617da2e4f3e83e1fc31938d609e
I20260807 00:43:20.026690 24345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:43:20.027922 24345 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:43:20.028156 24345 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Loaded TSK: 0
I20260807 00:43:20.040388 24345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Generated new TSK 1
I20260807 00:43:20.040494 24345 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:43:20.095417 24330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:20.095634 24330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:20.095672 24330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:20.096760 24330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:20.096830 24330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:20.096877 24330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:43:20.098320 24330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:35689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=43291
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:20.098611 24330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:20.098834 24330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:20.104017 24365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:20.104043 24364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:20.104276 24330 server_base.cc:1034] running on GCE node
W20260807 00:43:20.104398 24369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:20.104722 24330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:20.104986 24330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:20.106166 24330 hybrid_clock.cc:648] HybridClock initialized: now 1786063400106135 us; error 52 us; skew 500 ppm
Aug 07 00:43:20 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:20.109056 24330 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:43:20.109525 24330 webserver.cc:460] Webserver started at http://127.1.116.129:43291/ using document root <none> and password file <none>
I20260807 00:43:20.109776 24330 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:20.109838 24330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:20.111469 24330 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:43:20.112299 24377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.112566 24330 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:20.112689 24330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "409276227a7a4bb09bc09c5e364c48dd"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "6c70570bde0bf987e47ed39327d96661"
server_key_iv: "38200b2bbd83061dd902cd25f02f4c36"
server_key_version: "encryptionkey@0"
I20260807 00:43:20.113024 24330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:20.137614 24330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:20.139833 24330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:20.139991 24330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:20.140230 24330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:20.140731 24388 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:43:20.142138 24330 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:43:20.142195 24330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260807 00:43:20.142230 24330 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:43:20.143070 24330 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:43:20.143124 24330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 00:43:20.143153 24388 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap starting.
I20260807 00:43:20.151997 24330 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:35689
I20260807 00:43:20.152221 24543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:35689 every 8 connection(s)
I20260807 00:43:20.152550 24330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:43:20.155134 24388 log.cc:824] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Log is configured to *not* fsync() on all Append() calls
Aug 07 00:43:20 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:43:20.161686 24388 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:43:20.162075 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 24330
I20260807 00:43:20.162385 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:45141
--local_ip_for_outbound_sockets=127.1.116.130
--tserver_master_addrs=127.1.116.190:36907
--webserver_port=37209
--webserver_interface=127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:43:20.163849 24388 tablet_bootstrap.cc:492] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap complete.
I20260807 00:43:20.165153 24388 ts_tablet_manager.cc:1397] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.011s
I20260807 00:43:20.165723 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.153387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:58597)
0807 00:43:20.153847 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.153899 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.154649 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.155326 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.155331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.155742 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:20.156043 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.156050 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.157272 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.157274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.158754 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.158758 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.159447 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.159449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.159598 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.160133 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:20.160152 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:20.161628 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:20.163747 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.163755 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.163757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.164147 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.164435 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.164436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.164437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.164564 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.165038 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:20.165045 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:20.165192 (+ 147us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.165347 (+ 155us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.165521 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":129,"threads_started":1}
I20260807 00:43:20.166218 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.153892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:58597 (local address 127.1.116.190:36907)
0807 00:43:20.154687 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.154698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.154781 (+ 83us) server_negotiation.cc:407] Connection header received
0807 00:43:20.154929 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.154937 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.155022 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.155182 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:20.156178 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.157127 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.158864 (+ 1737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.159298 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.159421 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.161784 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:20.161789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:20.161789 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:20.161801 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:20.163633 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.164245 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.164248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.164249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.164360 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.164637 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.164638 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.164638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.164919 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:20.164973 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.165486 (+ 513us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.165685 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":110,"threads_started":1}
I20260807 00:43:20.166613 24388 raft_consensus.cc:348] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:43:20.166934 24544 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36907
I20260807 00:43:20.167026 24388 raft_consensus.cc:729] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409276227a7a4bb09bc09c5e364c48dd, State: Initialized, Role: FOLLOWER
I20260807 00:43:20.167078 24544 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:20.167196 24388 consensus_queue.cc:260] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } }
I20260807 00:43:20.167275 24544 heartbeater.cc:511] Master 127.1.116.190:36907 requested a full tablet report, sending...
I20260807 00:43:20.167472 24388 ts_tablet_manager.cc:1428] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260807 00:43:20.168254 24272 ts_manager.cc:194] Registered new tserver with Master: 409276227a7a4bb09bc09c5e364c48dd (127.1.116.129:35689)
I20260807 00:43:20.169480 24272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:58597
I20260807 00:43:20.170478 24544 heartbeater.cc:503] Master 127.1.116.190:36907 was elected leader, sending a full tablet report...
Aug 07 00:43:20 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063399, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.129@KRBTEST.COM
I20260807 00:43:20.179816 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.169352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35689 (local address 127.0.0.1:37072)
0807 00:43:20.169783 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.169822 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.169888 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.170226 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.170232 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.170650 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:20.170731 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.170734 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.172127 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.172128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.173087 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.173091 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.173807 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.173808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.173863 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.174406 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:20.174411 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:20.176208 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:20.178517 (+ 2309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.178525 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.178565 (+ 40us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.178801 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.179064 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.179065 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.179065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.179126 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.179481 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:20.179488 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:20.179529 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.179591 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.179739 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":80,"threads_started":1}
I20260807 00:43:20.179848 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.169609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37072 (local address 127.1.116.129:35689)
0807 00:43:20.170025 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.170030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.170039 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:43:20.170074 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.170076 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.170131 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.170820 (+ 689us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:20.170875 (+ 55us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.172014 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.173197 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.173711 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.173805 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.176314 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:20.176323 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:20.176323 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:20.176332 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:20.178418 (+ 2086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.178900 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.178903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.178903 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.178974 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.179217 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.179218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.179219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.179398 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:20.179513 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.179667 (+ 154us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.179750 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":70,"threads_started":1}
I20260807 00:43:20.180302 24449 tablet_service.cc:1514] Processing DeleteTablet for tablet 001b64164212429eb1aeb6378da490a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:42:59 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37072
I20260807 00:43:20.180934 24559 tablet_replica.cc:330] stopping tablet replica
I20260807 00:43:20.181057 24559 raft_consensus.cc:2229] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:43:20.181161 24559 raft_consensus.cc:2258] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:20.181530 24559 ts_tablet_manager.cc:1905] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:43:20.183771 24559 ts_tablet_manager.cc:1918] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:43:20.183847 24559 log.cc:1196] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal/wals/001b64164212429eb1aeb6378da490a6
I20260807 00:43:20.184093 24559 ts_tablet_manager.cc:1939] T 001b64164212429eb1aeb6378da490a6 P 409276227a7a4bb09bc09c5e364c48dd: Deleting consensus metadata
I20260807 00:43:20.184666 24250 catalog_manager.cc:4841] TS 409276227a7a4bb09bc09c5e364c48dd (127.1.116.129:35689): tablet 001b64164212429eb1aeb6378da490a6 (table test-table [id=3c967164d35b47f09561168df6b97032]) successfully deleted
W20260807 00:43:20.252597 24554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:20.252813 24554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:20.252851 24554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:20.253937 24554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:20.254001 24554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:20.254048 24554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:43:20.255565 24554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:45141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=37209
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:20.255820 24554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:20.256063 24554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:20.261380 24572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:20.261364 24569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:20.261682 24554 server_base.cc:1034] running on GCE node
W20260807 00:43:20.261364 24570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:20.261974 24554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:20.262261 24554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:20.263438 24554 hybrid_clock.cc:648] HybridClock initialized: now 1786063400263405 us; error 55 us; skew 500 ppm
Aug 07 00:43:20 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:20.265897 24554 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:43:20.266364 24554 webserver.cc:460] Webserver started at http://127.1.116.130:37209/ using document root <none> and password file <none>
I20260807 00:43:20.266649 24554 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:20.266712 24554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:20.268149 24554 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:43:20.268959 24581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.269325 24554 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:43:20.269436 24554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "2f54105b7a8641028afc0b021ba22361"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "b50ca90d0a3795ee71d18ee090caf4de"
server_key_iv: "2ffbf4052a28f561dd4b70dc46c7d21b"
server_key_version: "encryptionkey@0"
I20260807 00:43:20.269770 24554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:20.287042 24554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:20.288849 24554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:20.288959 24554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:20.289134 24554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:20.289439 24554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:20.289476 24554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.289549 24554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:20.289604 24554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.296574 24554 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:45141
I20260807 00:43:20.296720 24735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:45141 every 8 connection(s)
I20260807 00:43:20.297005 24554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:43:20.303476 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 24554
I20260807 00:43:20.303871 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:44575
--local_ip_for_outbound_sockets=127.1.116.131
--tserver_master_addrs=127.1.116.190:36907
--webserver_port=34283
--webserver_interface=127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:43:20 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:43:20.308470 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.297882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:47837 (local address 127.1.116.190:36907)
0807 00:43:20.297961 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.297963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.298328 (+ 365us) server_negotiation.cc:407] Connection header received
0807 00:43:20.299110 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.299112 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.299157 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.299252 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:20.300195 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.300942 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.301929 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.302139 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.302199 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.304959 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:20.304964 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:20.304964 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:20.304973 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:20.306755 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.307208 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.307210 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.307210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.307275 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.307594 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.307596 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.307596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.307749 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:20.307821 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.308301 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.308391 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:43:20.308749 24738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.297779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:47837)
0807 00:43:20.298160 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.298202 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.298968 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.299327 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.299330 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.299877 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:20.300057 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.300061 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.301055 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.301056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.301842 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.301844 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.302225 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.302226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.302325 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.302897 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:20.302906 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:20.304806 (+ 1900us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:20.306856 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.306863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.306866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.307121 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.307384 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.307385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.307385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.307505 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.307853 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:20.307861 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:20.308051 (+ 190us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.308235 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.308442 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":106,"threads_started":1}
I20260807 00:43:20.309238 24736 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36907
I20260807 00:43:20.309337 24736 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:20.309527 24736 heartbeater.cc:511] Master 127.1.116.190:36907 requested a full tablet report, sending...
I20260807 00:43:20.309887 24272 ts_manager.cc:194] Registered new tserver with Master: 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141)
I20260807 00:43:20.310289 24272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:47837
W20260807 00:43:20.397933 24739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:20.398169 24739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:20.398195 24739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:20.399327 24739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:20.399371 24739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:20.399398 24739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:43:20.400902 24739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=34283
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:20.401118 24739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:20.401356 24739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:20.407286 24748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:20.407356 24745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:20.407474 24739 server_base.cc:1034] running on GCE node
W20260807 00:43:20.407366 24744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:20.407821 24739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:20.408155 24739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:20.409309 24739 hybrid_clock.cc:648] HybridClock initialized: now 1786063400409295 us; error 34 us; skew 500 ppm
Aug 07 00:43:20 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:20.412405 24739 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:43:20.412899 24739 webserver.cc:460] Webserver started at http://127.1.116.131:34283/ using document root <none> and password file <none>
I20260807 00:43:20.413144 24739 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:20.413231 24739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:20.414840 24739 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:43:20.415623 24758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.415923 24739 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:20.416018 24739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "74235eae63044f5b955ed438796ab92b"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "f5950fbc4efb3657f58ef4c55d8d7599"
server_key_iv: "da083f10e860f692867731b1b62fb84a"
server_key_version: "encryptionkey@0"
I20260807 00:43:20.416496 24739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:20.437361 24739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:20.439720 24739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:20.439888 24739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:20.440181 24739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:20.440552 24739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:20.440590 24739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.440615 24739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:20.440662 24739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.448045 24739 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44575
I20260807 00:43:20.448238 24918 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44575 every 8 connection(s)
I20260807 00:43:20.448567 24739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:43:20.452297 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 24739
Aug 07 00:43:20 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:43:20.460618 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.452828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50310 (local address 127.1.116.190:36907)
0807 00:43:20.452983 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.452985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.453034 (+ 49us) server_negotiation.cc:407] Connection header received
0807 00:43:20.453101 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.453102 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.453116 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.453211 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:20.453872 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.454617 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.455466 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.455735 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.455797 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.457348 (+ 1551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:20.457366 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:20.457367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:20.457375 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:20.459329 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.459736 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.459737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.459738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.459792 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.460073 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.460074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.460074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.460213 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:20.460437 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.460469 (+ 32us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.460538 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":13,"server-negotiator.queue_time_us":128,"thread_start_us":78,"threads_started":1}
I20260807 00:43:20.460974 24549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.449480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:34847 (local address 127.1.116.190:36907)
0807 00:43:20.449567 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.449569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.449801 (+ 232us) server_negotiation.cc:407] Connection header received
0807 00:43:20.450437 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.450438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.450450 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.450598 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:20.451767 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.452376 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.453803 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.454044 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.454100 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.456948 (+ 2848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:20.456950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:20.456951 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:20.456957 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:20.459292 (+ 2335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.459843 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.459844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.459845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.459903 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.460206 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.460208 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.460208 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.460320 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:20.460386 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.460766 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.460874 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":49}
I20260807 00:43:20.461155 24923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.449444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:34847)
0807 00:43:20.449658 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.449701 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.450289 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.450687 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.450692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.451387 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:20.451666 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.451672 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.452513 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.452514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.453698 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.453701 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.454141 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.454142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.454295 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.454922 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:20.454933 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:20.456828 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:20.459418 (+ 2590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.459426 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.459428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.459762 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.459981 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:20.459982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:20.459983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:20.460133 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:20.460403 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:20.460411 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:20.460603 (+ 192us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.460726 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.460906 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":135,"threads_started":1}
I20260807 00:43:20.461725 24920 heartbeater.cc:347] Connected to a master server at 127.1.116.190:36907
I20260807 00:43:20.461826 24920 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:20.462158 24920 heartbeater.cc:511] Master 127.1.116.190:36907 requested a full tablet report, sending...
I20260807 00:43:20.462673 24272 ts_manager.cc:194] Registered new tserver with Master: 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
I20260807 00:43:20.463043 24272 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:34847
I20260807 00:43:20.472842 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:43:20.475787 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.473295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.1.116.190:36907)
0807 00:43:20.473367 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.473369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.473481 (+ 112us) server_negotiation.cc:407] Connection header received
0807 00:43:20.473540 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.473540 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.473580 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.473663 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:43:20.473794 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.474297 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.475158 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.475351 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.475411 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.475541 (+ 130us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:43:20.475593 (+ 52us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 00:43:20.475735 (+ 142us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50320: token expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260807 00:43:20.475855 24925 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50320: token expired
W20260807 00:43:20.476060 24171 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.116.190:36907)
I20260807 00:43:20.482436 24925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.476355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50324 (local address 127.1.116.190:36907)
0807 00:43:20.476390 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.476392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.476402 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:43:20.476432 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.476433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.476440 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.476486 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:20.476962 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.477411 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.478112 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.478321 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.478379 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.479793 (+ 1414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:20.479795 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:20.479795 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:20.479801 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:20.481406 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.481769 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.481770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.481770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.481818 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:20.482021 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:20.482022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:20.482022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:20.482116 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:20.482223 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.482311 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.482373 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260807 00:43:20.483908 24272 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:43:20.485936 24272 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:43:20.489924 24449 tablet_service.cc:1467] Processing CreateTablet for tablet 1135521dba1244569d3207e238e7ec7a (DEFAULT_TABLE table=test-table [id=d743da7792fd4570a0330ee2a89f8637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:20.490166 24449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1135521dba1244569d3207e238e7ec7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:20.492156 24388 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap starting.
I20260807 00:43:20.492579 24388 tablet_bootstrap.cc:654] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:20.493625 24388 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: No bootstrap required, opened a new log
I20260807 00:43:20.493697 24388 ts_tablet_manager.cc:1397] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:43:20.493777 24556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.488688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:45141 (local address 127.0.0.1:36624)
0807 00:43:20.488761 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.488775 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.488819 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.489453 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.489454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.489457 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:20.489544 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.489548 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.491128 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.491130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.492365 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.492369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.493478 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.493479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.493564 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.493589 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.493625 (+ 36us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.493686 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":3}
I20260807 00:43:20.493822 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.489629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48866 (local address 127.1.116.131:44575)
0807 00:43:20.489965 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.489971 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.490007 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:43:20.490053 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.490055 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.490100 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.490225 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:20.490674 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.491435 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.492484 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.493305 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.493391 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.493454 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.493694 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.493731 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":121,"threads_started":1}
I20260807 00:43:20.493873 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.488811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44575 (local address 127.0.0.1:48866)
0807 00:43:20.489005 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.489018 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.489085 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.490384 (+ 1299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.490385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.490388 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:20.490479 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.490482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.491517 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.491518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.492385 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.492387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.493395 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.493396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.493566 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.493590 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.493674 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.493730 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":1,"thread_start_us":119,"threads_started":1}
I20260807 00:43:20.493888 24388 raft_consensus.cc:348] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.493990 24388 raft_consensus.cc:374] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:20.494040 24388 raft_consensus.cc:729] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409276227a7a4bb09bc09c5e364c48dd, State: Initialized, Role: FOLLOWER
I20260807 00:43:20.494102 24388 consensus_queue.cc:260] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.494216 24388 ts_tablet_manager.cc:1428] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:20.494307 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.488968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36624 (local address 127.1.116.130:45141)
0807 00:43:20.489245 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.489251 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.489264 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:43:20.489313 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.489315 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.489335 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.489426 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:20.489636 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.491033 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.492457 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.493183 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.494055 (+ 872us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.494126 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.494155 (+ 29us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.494211 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":119,"threads_started":1}
I20260807 00:43:20.494756 24821 tablet_service.cc:1467] Processing CreateTablet for tablet 1135521dba1244569d3207e238e7ec7a (DEFAULT_TABLE table=test-table [id=d743da7792fd4570a0330ee2a89f8637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:20.495038 24642 tablet_service.cc:1467] Processing CreateTablet for tablet 1135521dba1244569d3207e238e7ec7a (DEFAULT_TABLE table=test-table [id=d743da7792fd4570a0330ee2a89f8637]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:20.495173 24821 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1135521dba1244569d3207e238e7ec7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:20.495431 24642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1135521dba1244569d3207e238e7ec7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:20.498271 24931 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Bootstrap starting.
I20260807 00:43:20.499099 24933 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Bootstrap starting.
I20260807 00:43:20.499198 24931 tablet_bootstrap.cc:654] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:20.499768 24933 tablet_bootstrap.cc:654] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:20.499783 24931 log.cc:824] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:20.500296 24933 log.cc:824] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:20.500658 24931 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: No bootstrap required, opened a new log
I20260807 00:43:20.500757 24931 ts_tablet_manager.cc:1397] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:43:20.501161 24933 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: No bootstrap required, opened a new log
I20260807 00:43:20.501256 24933 ts_tablet_manager.cc:1397] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:43:20.502990 24931 raft_consensus.cc:348] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.503206 24931 raft_consensus.cc:374] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:20.503248 24931 raft_consensus.cc:729] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74235eae63044f5b955ed438796ab92b, State: Initialized, Role: FOLLOWER
I20260807 00:43:20.503410 24931 consensus_queue.cc:260] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.503404 24933 raft_consensus.cc:348] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.503592 24933 raft_consensus.cc:374] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:20.503645 24933 raft_consensus.cc:729] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f54105b7a8641028afc0b021ba22361, State: Initialized, Role: FOLLOWER
I20260807 00:43:20.503788 24931 ts_tablet_manager.cc:1428] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:43:20.503765 24933 consensus_queue.cc:260] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.503832 24920 heartbeater.cc:503] Master 127.1.116.190:36907 was elected leader, sending a full tablet report...
I20260807 00:43:20.504031 24933 ts_tablet_manager.cc:1428] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:43:20.504184 24736 heartbeater.cc:503] Master 127.1.116.190:36907 was elected leader, sending a full tablet report...
W20260807 00:43:20.547750 24737 tablet.cc:2369] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:43:20.570703 24555 raft_consensus.cc:482] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:20.570856 24555 raft_consensus.cc:504] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.571286 24555 leader_election.cc:288] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141), 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
I20260807 00:43:20.574663 24929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.571671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:48071 (local address 127.1.116.131:44575)
0807 00:43:20.571776 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.571778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.571789 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:43:20.571865 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.571866 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.571878 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.571978 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:20.572160 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.572709 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.573811 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.574197 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.574300 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.574310 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.574518 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.574588 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 00:43:20.574663 24548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.571534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44575 (local address 127.1.116.129:48071)
0807 00:43:20.571649 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.571666 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.571764 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.571987 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.571988 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.571991 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:20.572079 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.572081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.572799 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.572800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.573705 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.573708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.574303 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.574305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.574401 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.574420 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.574508 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.574588 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 00:43:20.574844 24928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.571697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:35335 (local address 127.1.116.130:45141)
0807 00:43:20.571761 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.571765 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.571893 (+ 128us) server_negotiation.cc:407] Connection header received
0807 00:43:20.572011 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.572017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.572026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.572084 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:20.572306 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.572964 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.573966 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.574365 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.574456 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.574467 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.574718 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.574759 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:43:20.574873 24939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.571536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:45141 (local address 127.1.116.129:35335)
0807 00:43:20.571797 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.571809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:20.571914 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:20.572107 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:20.572108 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:20.572110 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:20.572221 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.572223 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.573047 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.573048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.573867 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:20.573869 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.574486 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.574487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:20.574607 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.574612 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:43:20.574701 (+ 89us) client_negotiation.cc:240] Negotiation successful
0807 00:43:20.574752 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"mutex_wait_us":19,"thread_start_us":144,"threads_started":1}
I20260807 00:43:20.575306 24667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1135521dba1244569d3207e238e7ec7a" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f54105b7a8641028afc0b021ba22361" is_pre_election: true
I20260807 00:43:20.575522 24667 raft_consensus.cc:2454] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 0.
I20260807 00:43:20.575873 24381 leader_election.cc:302] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f54105b7a8641028afc0b021ba22361, 409276227a7a4bb09bc09c5e364c48dd; no voters:
I20260807 00:43:20.575841 24848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1135521dba1244569d3207e238e7ec7a" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74235eae63044f5b955ed438796ab92b" is_pre_election: true
I20260807 00:43:20.576053 24848 raft_consensus.cc:2454] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 0.
I20260807 00:43:20.576181 24555 raft_consensus.cc:2781] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:43:20.576277 24555 raft_consensus.cc:482] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:43:20.576310 24555 raft_consensus.cc:3037] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:20.577135 24555 raft_consensus.cc:504] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.577298 24555 leader_election.cc:288] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 election: Requested vote from peers 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141), 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
I20260807 00:43:20.577492 24667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1135521dba1244569d3207e238e7ec7a" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f54105b7a8641028afc0b021ba22361"
I20260807 00:43:20.577528 24848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1135521dba1244569d3207e238e7ec7a" candidate_uuid: "409276227a7a4bb09bc09c5e364c48dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74235eae63044f5b955ed438796ab92b"
I20260807 00:43:20.577589 24667 raft_consensus.cc:3037] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:20.577631 24848 raft_consensus.cc:3037] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:20.578336 24667 raft_consensus.cc:2454] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 1.
I20260807 00:43:20.578562 24381 leader_election.cc:302] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f54105b7a8641028afc0b021ba22361, 409276227a7a4bb09bc09c5e364c48dd; no voters:
I20260807 00:43:20.578722 24555 raft_consensus.cc:2781] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:43:20.578761 24848 raft_consensus.cc:2454] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 409276227a7a4bb09bc09c5e364c48dd in term 1.
I20260807 00:43:20.578923 24555 raft_consensus.cc:686] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 LEADER]: Becoming Leader. State: Replica: 409276227a7a4bb09bc09c5e364c48dd, State: Running, Role: LEADER
I20260807 00:43:20.579020 24555 consensus_queue.cc:237] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:20.579881 24272 catalog_manager.cc:5495] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 409276227a7a4bb09bc09c5e364c48dd (127.1.116.129). New cstate: current_term: 1 leader_uuid: "409276227a7a4bb09bc09c5e364c48dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } health_report { overall_health: HEALTHY } } }
I20260807 00:43:20.587371 24558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.584763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.1.116.129:35689)
0807 00:43:20.584830 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:20.584832 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:20.584840 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:43:20.584867 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:20.584868 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:20.584885 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:20.584952 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:43:20.585244 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.585718 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.586486 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:20.586769 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:20.586831 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:20.587010 (+ 179us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:43:20.587097 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:43:20.587157 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:20.587249 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 00:43:20.587269 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:43:20.588960 24667 raft_consensus.cc:1261] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Refusing update from remote peer 409276227a7a4bb09bc09c5e364c48dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:43:20.588986 24848 raft_consensus.cc:1261] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Refusing update from remote peer 409276227a7a4bb09bc09c5e364c48dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:43:20.589413 24940 consensus_queue.cc:1035] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:43:20.589555 24555 consensus_queue.cc:1035] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:43:20.592171 24943 mvcc.cc:205] Tried to move back new op lower bound from 7315715688809418752 to 7315715688772227072. Current Snapshot: MvccSnapshot[applied={T|T < 7315715688809418752}]
I20260807 00:43:20.592211 24944 mvcc.cc:205] Tried to move back new op lower bound from 7315715688809418752 to 7315715688772227072. Current Snapshot: MvccSnapshot[applied={T|T < 7315715688809418752}]
I20260807 00:43:20.593281 24946 mvcc.cc:205] Tried to move back new op lower bound from 7315715688809418752 to 7315715688772227072. Current Snapshot: MvccSnapshot[applied={T|T < 7315715688809418752}]
I20260807 00:43:20.595419 24271 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:43:20.595520 24271 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:43:20.596346 24271 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Sending DeleteTablet for 3 replicas of tablet 1135521dba1244569d3207e238e7ec7a
I20260807 00:43:20.596675 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 24330
I20260807 00:43:20.596683 24642 tablet_service.cc:1514] Processing DeleteTablet for tablet 1135521dba1244569d3207e238e7ec7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:20 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:36624
I20260807 00:43:20.596814 24821 tablet_service.cc:1514] Processing DeleteTablet for tablet 1135521dba1244569d3207e238e7ec7a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:20 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:48866
I20260807 00:43:20.602100 24963 tablet_replica.cc:330] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: stopping tablet replica
I20260807 00:43:20.602216 24963 raft_consensus.cc:2229] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:43:20.602321 24963 raft_consensus.cc:2258] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:20.604205 24959 tablet_replica.cc:330] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: stopping tablet replica
I20260807 00:43:20.604310 24959 raft_consensus.cc:2229] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:43:20.604408 24959 raft_consensus.cc:2258] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:20.604581 24963 ts_tablet_manager.cc:1905] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:43:20.607017 24963 ts_tablet_manager.cc:1918] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:43:20.607170 24963 log.cc:1196] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/1135521dba1244569d3207e238e7ec7a
I20260807 00:43:20.607326 24959 ts_tablet_manager.cc:1905] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:43:20.607666 24963 ts_tablet_manager.cc:1939] T 1135521dba1244569d3207e238e7ec7a P 74235eae63044f5b955ed438796ab92b: Deleting consensus metadata
I20260807 00:43:20.608315 24251 catalog_manager.cc:4841] TS 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575): tablet 1135521dba1244569d3207e238e7ec7a (table test-table [id=d743da7792fd4570a0330ee2a89f8637]) successfully deleted
I20260807 00:43:20.609369 24959 ts_tablet_manager.cc:1918] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:43:20.609447 24959 log.cc:1196] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/1135521dba1244569d3207e238e7ec7a
I20260807 00:43:20.609758 24959 ts_tablet_manager.cc:1939] T 1135521dba1244569d3207e238e7ec7a P 2f54105b7a8641028afc0b021ba22361: Deleting consensus metadata
I20260807 00:43:20.610212 24253 catalog_manager.cc:4841] TS 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141): tablet 1135521dba1244569d3207e238e7ec7a (table test-table [id=d743da7792fd4570a0330ee2a89f8637]) successfully deleted
W20260807 00:43:20.613336 24250 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:35689 (error 108)
I20260807 00:43:20.613622 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 24554
I20260807 00:43:20.617236 24927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:20.613787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35689 (local address 127.0.0.1:37080)
0807 00:43:20.617133 (+ 3346us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:20.617188 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:35689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3318}
W20260807 00:43:20.617487 24250 catalog_manager.cc:4584] TS 409276227a7a4bb09bc09c5e364c48dd (127.1.116.129:35689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1135521dba1244569d3207e238e7ec7a: Network error: Client connection negotiation failed: client connection to 127.1.116.129:35689: connect: Connection refused (error 111)
I20260807 00:43:20.622781 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 24739
I20260807 00:43:20.634070 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 24231
I20260807 00:43:40.643827 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36907
--webserver_interface=127.1.116.190
--webserver_port=40267
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:36907
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:40.741837 24965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:40.742074 24965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:40.742105 24965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:40.743263 24965 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 00:43:40.743299 24965 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 00:43:40.743311 24965 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 00:43:40.743323 24965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:43:40.743336 24965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:40.743356 24965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:40.743368 24965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:43:40.743381 24965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:43:40.744892 24965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:36907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:36907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=40267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24965
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:40.745131 24965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:40.745333 24965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:40.753088 24977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:40.753075 24974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:40.753077 24973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:40.753233 24965 server_base.cc:1034] running on GCE node
I20260807 00:43:40.753715 24965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:40.754043 24965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:40.755232 24965 hybrid_clock.cc:648] HybridClock initialized: now 1786063420755219 us; error 28 us; skew 500 ppm
Aug 07 00:43:40 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063420, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:40.757987 24965 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:43:40.758419 24965 webserver.cc:460] Webserver started at http://127.1.116.190:40267/ using document root <none> and password file <none>
I20260807 00:43:40.758715 24965 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:40.758770 24965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:40.760401 24965 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:40.761401 24990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:40.761693 24965 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:40.761806 24965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "8c8d0e67e9a941ed973e021b09adc455"
format_stamp: "Formatted at 2026-08-07 00:42:58 on dist-test-slave-h6n0"
server_key: "91568299d997094718f16a5f81ad035e"
server_key_iv: "348a8fb47a2e6f6ab774fcd2075ccd5e"
server_key_version: "encryptionkey@0"
I20260807 00:43:40.762200 24965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:40.794873 24965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:40.797405 24965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:40.797577 24965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:40.802975 24965 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:36907
I20260807 00:43:40.803087 25064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:36907 every 8 connection(s)
I20260807 00:43:40.803637 24965 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:43:40.805272 25065 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:43:40.806118 25065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Bootstrap starting.
I20260807 00:43:40.812530 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 24965
I20260807 00:43:40.812924 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:35689
--local_ip_for_outbound_sockets=127.1.116.129
--tserver_master_addrs=127.1.116.190:36907
--webserver_port=43291
--webserver_interface=127.1.116.129
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:43:40.818341 25065 log.cc:824] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:40.820951 25065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:43:40.821322 25065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Bootstrap complete.
I20260807 00:43:40.824021 25065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:40.824280 25065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c8d0e67e9a941ed973e021b09adc455, State: Initialized, Role: FOLLOWER
I20260807 00:43:40.824476 25065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:40.824569 25065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:43:40.824640 25065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:43:40.824715 25065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 2 FOLLOWER]: Advancing to term 3
I20260807 00:43:40.825862 25065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:40.826006 25065 leader_election.cc:302] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c8d0e67e9a941ed973e021b09adc455; no voters:
I20260807 00:43:40.826231 25065 leader_election.cc:288] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 00:43:40.826272 25071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 00:43:40.826433 25071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [term 3 LEADER]: Becoming Leader. State: Replica: 8c8d0e67e9a941ed973e021b09adc455, State: Running, Role: LEADER
I20260807 00:43:40.826581 25065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:43:40.826558 25071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } }
I20260807 00:43:40.826998 25074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8c8d0e67e9a941ed973e021b09adc455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } } }
I20260807 00:43:40.827088 25074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:40.827257 25075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c8d0e67e9a941ed973e021b09adc455. Latest consensus state: current_term: 3 leader_uuid: "8c8d0e67e9a941ed973e021b09adc455" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c8d0e67e9a941ed973e021b09adc455" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 36907 } } }
I20260807 00:43:40.827342 25075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:40.827911 25077 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:43:40.828904 25077 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:43:40.829125 25077 catalog_manager.cc:1259] Loaded cluster ID: 69a69617da2e4f3e83e1fc31938d609e
I20260807 00:43:40.829418 25077 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:43:40.830752 25077 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:43:40.830978 25077 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Loaded TSK: 0
I20260807 00:43:40.831034 25077 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8c8d0e67e9a941ed973e021b09adc455: Loaded TSK: 1
F20260807 00:43:40.831284 25077 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ff47a4569ad google::LogMessage::Fail() at ??:0
@ 0x7ff47a45988a google::LogMessage::SendToLog() at ??:0
@ 0x7ff47a4564de google::LogMessage::Flush() at ??:0
@ 0x7ff47a457a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ff47cbc3104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ff47cbc0d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ff4823b01cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ff4823b048e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ff4823b0594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ff482376c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7ff482380ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ff47c3240d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ff47c319954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ff47d17d6db start_thread at ??:0
@ 0x7ff478f5461f clone at ??:0
W20260807 00:43:40.920094 25069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:40.920284 25069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:40.920306 25069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:40.921430 25069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:40.921475 25069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:40.921501 25069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:43:40.923025 25069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:35689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=43291
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:40.923264 25069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:40.923530 25069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:40.928754 25132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:40.928807 25128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:40.928977 25069 server_base.cc:1034] running on GCE node
W20260807 00:43:40.928753 25129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:40.929348 25069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:40.929649 25069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:40.930804 25069 hybrid_clock.cc:648] HybridClock initialized: now 1786063420930773 us; error 49 us; skew 500 ppm
Aug 07 00:43:40 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063420, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:40.933298 25069 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:43:40.933774 25069 webserver.cc:460] Webserver started at http://127.1.116.129:43291/ using document root <none> and password file <none>
I20260807 00:43:40.934029 25069 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:40.934103 25069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:40.935475 25069 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:40.936260 25142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:40.936650 25069 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:40.936801 25069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "409276227a7a4bb09bc09c5e364c48dd"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "6c70570bde0bf987e47ed39327d96661"
server_key_iv: "38200b2bbd83061dd902cd25f02f4c36"
server_key_version: "encryptionkey@0"
I20260807 00:43:40.937144 25069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:40.944803 25069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:40.947224 25069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:40.947369 25069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:40.947616 25069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:40.948113 25152 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:43:40.950454 25069 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:43:40.950520 25069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260807 00:43:40.950583 25069 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:43:40.951601 25069 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:43:40.951654 25069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260807 00:43:40.951773 25152 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap starting.
I20260807 00:43:40.961074 25069 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:35689
I20260807 00:43:40.961226 25309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:35689 every 8 connection(s)
I20260807 00:43:40.961750 25069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:43:40.963806 25152 log.cc:824] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:40.966012 25152 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:43:40.966320 25152 tablet_bootstrap.cc:492] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Bootstrap complete.
I20260807 00:43:40.967346 25152 ts_tablet_manager.cc:1397] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.012s
I20260807 00:43:40.968401 25152 raft_consensus.cc:348] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:40.968624 25152 raft_consensus.cc:729] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 409276227a7a4bb09bc09c5e364c48dd, State: Initialized, Role: FOLLOWER
I20260807 00:43:40.968763 25152 consensus_queue.cc:260] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:40.969069 25152 ts_tablet_manager.cc:1428] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 00:43:40.970950 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 25069
I20260807 00:43:40.971344 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:45141
--local_ip_for_outbound_sockets=127.1.116.130
--tserver_master_addrs=127.1.116.190:36907
--webserver_port=37209
--webserver_interface=127.1.116.130
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:41.040936 25316 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:43:40.962852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:58613)
0807 00:43:40.963147 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:40.963207 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:40.963871 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:41.040642 (+ 76771us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":151,"threads_started":1}
W20260807 00:43:41.041232 25314 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:36907 (0 consecutive failures): Network error: Failed to ping master at 127.1.116.190:36907: Client connection negotiation failed: client connection to 127.1.116.190:36907: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:43:41.041659 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.041522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:53301)
0807 00:43:41.041588 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.041636 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 00:43:41.042033 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.041941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:56927)
0807 00:43:41.041986 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.042010 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 00:43:41.042188 25314 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:43:41.057178 25322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:41.057410 25322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:41.057448 25322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:41.058663 25322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:41.058727 25322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:41.058771 25322 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:43:41.060678 25322 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:45141
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=37209
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:41.060951 25322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:41.061180 25322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:41.066344 25339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:41.066401 25334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:41.066401 25328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:41.066931 25322 server_base.cc:1034] running on GCE node
I20260807 00:43:41.067198 25322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:41.067498 25322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:41.068682 25322 hybrid_clock.cc:648] HybridClock initialized: now 1786063421068667 us; error 31 us; skew 500 ppm
Aug 07 00:43:41 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063421, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:41.071413 25322 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:43:41.071889 25322 webserver.cc:460] Webserver started at http://127.1.116.130:37209/ using document root <none> and password file <none>
I20260807 00:43:41.072127 25322 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:41.072196 25322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:41.073681 25322 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:43:41.074349 25348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:41.074608 25322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:43:41.074715 25322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "2f54105b7a8641028afc0b021ba22361"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "b50ca90d0a3795ee71d18ee090caf4de"
server_key_iv: "2ffbf4052a28f561dd4b70dc46c7d21b"
server_key_version: "encryptionkey@0"
I20260807 00:43:41.075060 25322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:41.100865 25322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:41.103204 25322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:41.103367 25322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:41.103614 25322 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:41.103932 25322 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:41.103988 25322 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:41.104033 25322 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:41.104075 25322 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:41.111321 25322 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:45141
I20260807 00:43:41.111538 25512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:45141 every 8 connection(s)
I20260807 00:43:41.111733 25322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:43:41.113042 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.112611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:46885)
0807 00:43:41.112816 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.112892 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
W20260807 00:43:41.113653 25513 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:36907 (0 consecutive failures): Network error: Failed to ping master at 127.1.116.190:36907: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
I20260807 00:43:41.114022 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.113908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:38599)
0807 00:43:41.113970 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.114002 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 00:43:41.114455 25516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.114369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:41497)
0807 00:43:41.114416 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.114440 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 00:43:41.114660 25513 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:43:41.118741 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 25322
I20260807 00:43:41.119082 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:44575
--local_ip_for_outbound_sockets=127.1.116.131
--tserver_master_addrs=127.1.116.190:36907
--webserver_port=34283
--webserver_interface=127.1.116.131
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:41.211956 25517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:41.212186 25517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:41.212210 25517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:41.213289 25517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:41.213328 25517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:41.213353 25517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:43:41.214809 25517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:44575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=34283
--tserver_master_addrs=127.1.116.190:36907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:41.215013 25517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:41.215214 25517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:41.221045 25528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:41.221019 25523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:41.221019 25524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:41.221549 25517 server_base.cc:1034] running on GCE node
I20260807 00:43:41.221756 25517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:41.222007 25517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:41.223191 25517 hybrid_clock.cc:648] HybridClock initialized: now 1786063421223183 us; error 49 us; skew 500 ppm
Aug 07 00:43:41 dist-test-slave-h6n0 krb5kdc[23492](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063421, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:41.226199 25517 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:43:41.226831 25517 webserver.cc:460] Webserver started at http://127.1.116.131:34283/ using document root <none> and password file <none>
I20260807 00:43:41.227088 25517 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:41.227162 25517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:41.228729 25517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:43:41.229511 25540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:41.229743 25517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:41.229861 25517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "74235eae63044f5b955ed438796ab92b"
format_stamp: "Formatted at 2026-08-07 00:42:59 on dist-test-slave-h6n0"
server_key: "f5950fbc4efb3657f58ef4c55d8d7599"
server_key_iv: "da083f10e860f692867731b1b62fb84a"
server_key_version: "encryptionkey@0"
I20260807 00:43:41.230315 25517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:41.239979 25517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:41.242172 25517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:41.242352 25517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:41.242679 25517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:41.243057 25517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:41.243114 25517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:41.243157 25517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:41.243184 25517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:41.250748 25517 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44575
I20260807 00:43:41.250886 25706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44575 every 8 connection(s)
I20260807 00:43:41.251277 25517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:43:41.252630 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.252100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:47715)
0807 00:43:41.252369 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.252461 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":184,"threads_started":1}
W20260807 00:43:41.253166 25707 heartbeater.cc:644] Failed to heartbeat to 127.1.116.190:36907 (0 consecutive failures): Network error: Failed to ping master at 127.1.116.190:36907: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
I20260807 00:43:41.253510 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.253419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:45649)
0807 00:43:41.253461 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.253490 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260807 00:43:41.253854 25709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:41.253764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:58669)
0807 00:43:41.253815 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:41.253838 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 00:43:41.253995 25707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:43:41.256280 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 25517
I20260807 00:43:42.042981 25711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:42.042670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:42753)
0807 00:43:42.042835 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:42.042887 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":69,"threads_started":1}
I20260807 00:43:42.115224 25712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:42.114965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:33567)
0807 00:43:42.115111 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:42.115151 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260807 00:43:42.254768 25713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:42.254392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:42257)
0807 00:43:42.254632 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:42.254690 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":75,"threads_started":1}
I20260807 00:43:42.433106 25714 raft_consensus.cc:482] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:42.433426 25714 raft_consensus.cc:504] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:42.433920 25714 leader_election.cc:288] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141), 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
Aug 07 00:43:42 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063420, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.131@KRBTEST.COM
Aug 07 00:43:42 dist-test-slave-h6n0 krb5kdc[23492](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063420, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.130@KRBTEST.COM
I20260807 00:43:42.445691 25718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:42.434222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:46523 (local address 127.1.116.130:45141)
0807 00:43:42.434386 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:42.434400 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:42.434447 (+ 47us) server_negotiation.cc:407] Connection header received
0807 00:43:42.435148 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:42.435151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:42.435211 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:42.435371 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:42.436320 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.437386 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.438248 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.438825 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.438964 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:42.441887 (+ 2923us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:42.441893 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:42.441893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:42.441907 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:42.443955 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444420 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:42.444424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:42.444424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:42.444528 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444803 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:42.444804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:42.444804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:42.445048 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:42.445132 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:42.445435 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 00:43:42.445574 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":95,"threads_started":1}
I20260807 00:43:42.445683 25711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:42.434110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:45141 (local address 127.1.116.129:46523)
0807 00:43:42.434183 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:42.434195 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:42.434260 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:42.435458 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:42.435463 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:42.435923 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:42.436150 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:42.436160 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.437506 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.437508 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:42.438145 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:42.438146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.438922 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.438923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:42.438989 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:42.439747 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:42.439753 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:42.441778 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:42.444051 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444059 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:42.444061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:42.444333 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:42.444610 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:42.444612 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:42.444725 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:42.445129 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:42.445137 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:42.445268 (+ 131us) client_negotiation.cc:769] Sending connection context
0807 00:43:42.445371 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 00:43:42.445573 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":2}
I20260807 00:43:42.445905 25719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:42.434340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:43375 (local address 127.1.116.131:44575)
0807 00:43:42.434571 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:42.434583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:42.434618 (+ 35us) server_negotiation.cc:407] Connection header received
0807 00:43:42.435148 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:42.435151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:42.435211 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:42.435371 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:42.436275 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.437176 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.438245 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.438723 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.438857 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:42.441473 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:42.441480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:42.441480 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:42.441497 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:42.443963 (+ 2466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444513 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:42.444518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:42.444518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:42.444631 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444889 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:42.444890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:42.444890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:42.445222 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:42.445324 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:42.445555 (+ 231us) server_negotiation.cc:299] Negotiation successful
0807 00:43:42.445739 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":72,"threads_started":1}
I20260807 00:43:42.446697 25717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:42.434210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44575 (local address 127.1.116.129:43375)
0807 00:43:42.434373 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:42.434413 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:42.434484 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:42.435520 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:42.435523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:42.435931 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:42.436150 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:42.436156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.437289 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.437290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:42.438138 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:42.438140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:42.438832 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:42.438834 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:42.438971 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:42.439565 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:42.439576 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:42.441303 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:42.444100 (+ 2797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444102 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:42.444103 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:42.444396 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:42.444731 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:42.444732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:42.444732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:42.444813 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:42.445317 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:42.445319 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:42.445376 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 00:43:42.445476 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 00:43:42.445682 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":61,"thread_start_us":79,"threads_started":1}
W20260807 00:43:42.446921 25147 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
W20260807 00:43:42.447664 25146 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
I20260807 00:43:42.447748 25146 leader_election.cc:302] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 409276227a7a4bb09bc09c5e364c48dd; no voters: 2f54105b7a8641028afc0b021ba22361, 74235eae63044f5b955ed438796ab92b
I20260807 00:43:42.447916 25714 raft_consensus.cc:2726] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:43:43.043881 25720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:43.043578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:51185)
0807 00:43:43.043744 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:43.043800 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":79,"threads_started":1}
I20260807 00:43:43.116061 25721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:43.115828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:51889)
0807 00:43:43.115977 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:43.116020 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":68,"threads_started":1}
I20260807 00:43:43.255710 25722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:43.255387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:47047)
0807 00:43:43.255587 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:43.255648 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":89,"threads_started":1}
I20260807 00:43:44.044772 25724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:44.044474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:33115)
0807 00:43:44.044665 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:44.044708 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":98,"threads_started":1}
I20260807 00:43:44.097546 25725 raft_consensus.cc:482] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:44.097697 25725 raft_consensus.cc:504] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:44.097958 25725 leader_election.cc:288] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141), 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
W20260807 00:43:44.098450 25146 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
W20260807 00:43:44.098565 25147 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
I20260807 00:43:44.098701 25146 leader_election.cc:302] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 409276227a7a4bb09bc09c5e364c48dd; no voters: 2f54105b7a8641028afc0b021ba22361, 74235eae63044f5b955ed438796ab92b
I20260807 00:43:44.098850 25725 raft_consensus.cc:2726] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:43:44.116954 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:44.116700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:37359)
0807 00:43:44.116874 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:44.116909 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":115,"threads_started":1}
I20260807 00:43:44.256534 25728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:44.256273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:53847)
0807 00:43:44.256460 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:44.256498 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":89,"threads_started":1}
I20260807 00:43:45.045578 25730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:45.045304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:49955)
0807 00:43:45.045476 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:45.045517 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":86,"threads_started":1}
I20260807 00:43:45.117735 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:45.117496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:40465)
0807 00:43:45.117641 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:45.117686 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260807 00:43:45.257372 25733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:45.257091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:52713)
0807 00:43:45.257295 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:45.257332 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":113,"threads_started":1}
I20260807 00:43:46.046605 25734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:46.046195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:45261)
0807 00:43:46.046416 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:46.046479 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":82,"threads_started":1}
I20260807 00:43:46.118611 25735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:46.118267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:55073)
0807 00:43:46.118446 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:46.118498 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260807 00:43:46.258239 25736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:46.257888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:51891)
0807 00:43:46.258105 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:46.258181 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
I20260807 00:43:46.821147 25738 raft_consensus.cc:482] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:46.821275 25738 raft_consensus.cc:504] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:46.821524 25738 leader_election.cc:288] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141), 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
W20260807 00:43:46.821946 25147 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
W20260807 00:43:46.822036 25146 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
I20260807 00:43:46.822189 25146 leader_election.cc:302] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 409276227a7a4bb09bc09c5e364c48dd; no voters: 2f54105b7a8641028afc0b021ba22361, 74235eae63044f5b955ed438796ab92b
I20260807 00:43:46.822315 25738 raft_consensus.cc:2726] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:43:47.047531 25740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:47.047233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:47311)
0807 00:43:47.047433 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:47.047469 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":126,"threads_started":1}
I20260807 00:43:47.119500 25741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:47.119194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:34343)
0807 00:43:47.119390 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:47.119446 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":80,"threads_started":1}
I20260807 00:43:47.259153 25742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:47.258859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:54303)
0807 00:43:47.259048 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:47.259100 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
I20260807 00:43:48.048367 25743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:48.048086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:59349)
0807 00:43:48.048250 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:48.048302 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":75,"threads_started":1}
I20260807 00:43:48.120391 25745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:48.120068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:55667)
0807 00:43:48.120267 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:48.120311 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":116,"threads_started":1}
I20260807 00:43:48.260255 25746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:48.259823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:38117)
0807 00:43:48.260107 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:48.260171 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":115,"threads_started":1}
I20260807 00:43:49.049243 25748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:49.048968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:49487)
0807 00:43:49.049132 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:49.049178 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":76,"threads_started":1}
I20260807 00:43:49.121232 25750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:49.120935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:33943)
0807 00:43:49.121114 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:49.121167 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
I20260807 00:43:49.261276 25751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:49.260896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:43419)
0807 00:43:49.261148 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:49.261217 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":116,"threads_started":1}
I20260807 00:43:50.050089 25752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:50.049809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:57473)
0807 00:43:50.049994 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:50.050038 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
I20260807 00:43:50.122227 25753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:50.121938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:54011)
0807 00:43:50.122110 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:50.122163 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":72,"threads_started":1}
I20260807 00:43:50.262146 25754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:50.261841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:55095)
0807 00:43:50.262039 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:50.262095 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":107,"threads_started":1}
I20260807 00:43:51.051275 25755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:51.050838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:45479)
0807 00:43:51.051100 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:51.051221 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":108,"threads_started":1}
I20260807 00:43:51.123072 25756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:51.122812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:48685)
0807 00:43:51.122961 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:51.123014 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260807 00:43:51.221907 25757 raft_consensus.cc:482] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:51.222043 25757 raft_consensus.cc:504] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:51.222316 25757 leader_election.cc:288] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141), 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
W20260807 00:43:51.222837 25147 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
W20260807 00:43:51.223098 25146 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
I20260807 00:43:51.223233 25146 leader_election.cc:302] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 409276227a7a4bb09bc09c5e364c48dd; no voters: 2f54105b7a8641028afc0b021ba22361, 74235eae63044f5b955ed438796ab92b
I20260807 00:43:51.223348 25757 raft_consensus.cc:2726] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:43:51.262989 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:51.262723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:40983)
0807 00:43:51.262883 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:51.262923 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":80,"threads_started":1}
I20260807 00:43:52.052227 25759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:52.051933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:50991)
0807 00:43:52.052125 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:52.052164 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":112,"threads_started":1}
I20260807 00:43:52.123915 25760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:52.123575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:46525)
0807 00:43:52.123784 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:52.123851 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":86,"threads_started":1}
I20260807 00:43:52.263839 25761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:52.263551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:48219)
0807 00:43:52.263725 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:52.263774 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260807 00:43:53.053077 25762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:53.052778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:60247)
0807 00:43:53.052962 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:53.053025 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260807 00:43:53.124795 25763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:53.124538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:39519)
0807 00:43:53.124693 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:53.124737 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260807 00:43:53.264804 25764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:53.264486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:47659)
0807 00:43:53.264694 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:53.264742 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":109,"threads_started":1}
I20260807 00:43:53.517663 25765 raft_consensus.cc:482] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:53.517792 25765 raft_consensus.cc:504] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f54105b7a8641028afc0b021ba22361" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 45141 } } peers { permanent_uuid: "74235eae63044f5b955ed438796ab92b" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44575 } } peers { permanent_uuid: "409276227a7a4bb09bc09c5e364c48dd" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35689 } }
I20260807 00:43:53.517975 25765 leader_election.cc:288] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141), 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575)
W20260807 00:43:53.518477 25147 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f54105b7a8641028afc0b021ba22361 (127.1.116.130:45141): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
W20260807 00:43:53.518599 25146 leader_election.cc:341] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 74235eae63044f5b955ed438796ab92b (127.1.116.131:44575): Not found: Tablet not found: 1135521dba1244569d3207e238e7ec7a
I20260807 00:43:53.518667 25146 leader_election.cc:302] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 409276227a7a4bb09bc09c5e364c48dd; no voters: 2f54105b7a8641028afc0b021ba22361, 74235eae63044f5b955ed438796ab92b
I20260807 00:43:53.518769 25765 raft_consensus.cc:2726] T 1135521dba1244569d3207e238e7ec7a P 409276227a7a4bb09bc09c5e364c48dd [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:43:54.053979 25766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:54.053687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:50355)
0807 00:43:54.053869 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:54.053918 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":85,"threads_started":1}
I20260807 00:43:54.125509 25767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:54.125311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:41173)
0807 00:43:54.125431 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:54.125468 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":73,"threads_started":1}
I20260807 00:43:54.265583 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:54.265354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:39323)
0807 00:43:54.265510 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:54.265545 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260807 00:43:55.054910 25770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:55.054621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:50295)
0807 00:43:55.054820 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:55.054861 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":115,"threads_started":1}
I20260807 00:43:55.126410 25771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:55.126096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:52349)
0807 00:43:55.126295 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:55.126350 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":72,"threads_started":1}
I20260807 00:43:55.266479 25772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:55.266196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:50979)
0807 00:43:55.266357 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:55.266414 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":73,"threads_started":1}
I20260807 00:43:56.055852 25773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:56.055556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.129:35809)
0807 00:43:56.055759 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:56.055806 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":126,"threads_started":1}
I20260807 00:43:56.127280 25774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:56.127045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.130:56991)
0807 00:43:56.127201 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:56.127242 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":85,"threads_started":1}
I20260807 00:43:56.259064 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 25069
I20260807 00:43:56.267653 25775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:56.267360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:36907 (local address 127.1.116.131:57725)
0807 00:43:56.267536 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:56.267594 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.190:36907: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":88,"threads_started":1}
I20260807 00:43:56.272964 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 25322
I20260807 00:43:56.285179 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 25517
2026-08-07T00:43:56Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58847 ms)
[----------] 30 tests from SecurityITest (245907 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:43:56 dist-test-slave-h6n0 krb5kdc[25778](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:43:56 dist-test-slave-h6n0 krb5kdc[25778](info): set up 2 sockets
Aug 07 00:43:56 dist-test-slave-h6n0 krb5kdc[25778](info): commencing operation
krb5kdc: starting...
W20260807 00:43:57.561722 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:43:57 dist-test-slave-h6n0 krb5kdc[25778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:43:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:43:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:43:57.695271 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35165
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:35165
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:57.785977 25795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:57.786248 25795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:57.786284 25795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:57.787454 25795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:43:57.787516 25795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:57.787542 25795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:57.787571 25795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:43:57.787602 25795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:43:57.789162 25795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:35165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:35165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:57.789469 25795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:57.789726 25795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:57.795342 25800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:57.795361 25807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:57.795364 25801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:57.795440 25795 server_base.cc:1034] running on GCE node
I20260807 00:43:57.795975 25795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:57.796396 25795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:57.797588 25795 hybrid_clock.cc:648] HybridClock initialized: now 1786063437797571 us; error 57 us; skew 500 ppm
Aug 07 00:43:57 dist-test-slave-h6n0 krb5kdc[25778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063437, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:57.800606 25795 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:43:57.801203 25795 webserver.cc:460] Webserver started at http://127.1.116.190:40065/ using document root <none> and password file <none>
I20260807 00:43:57.801468 25795 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:57.801542 25795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:57.801717 25795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:57.803051 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "7a8421fa89d9481f8cb86aad80131a73"
format_stamp: "Formatted at 2026-08-07 00:43:57 on dist-test-slave-h6n0"
server_key: "ea9dc2364412abc255c6c6f3369b74cc"
server_key_iv: "9e650d859115bfda7b65f8332458d6f5"
server_key_version: "encryptionkey@0"
I20260807 00:43:57.803478 25795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "7a8421fa89d9481f8cb86aad80131a73"
format_stamp: "Formatted at 2026-08-07 00:43:57 on dist-test-slave-h6n0"
server_key: "ea9dc2364412abc255c6c6f3369b74cc"
server_key_iv: "9e650d859115bfda7b65f8332458d6f5"
server_key_version: "encryptionkey@0"
I20260807 00:43:57.805018 25795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:43:57.805900 25817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:57.806116 25795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:57.806238 25795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "7a8421fa89d9481f8cb86aad80131a73"
format_stamp: "Formatted at 2026-08-07 00:43:57 on dist-test-slave-h6n0"
server_key: "ea9dc2364412abc255c6c6f3369b74cc"
server_key_iv: "9e650d859115bfda7b65f8332458d6f5"
server_key_version: "encryptionkey@0"
I20260807 00:43:57.806322 25795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:57.816880 25795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:57.819039 25795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:57.819224 25795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:57.824720 25795 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:35165
I20260807 00:43:57.824775 25885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:35165 every 8 connection(s)
I20260807 00:43:57.825223 25795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:43:57.826305 25886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:57.828967 25886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73: Bootstrap starting.
I20260807 00:43:57.829562 25886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:57.829811 25886 log.cc:824] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:57.830678 25886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73: No bootstrap required, opened a new log
I20260807 00:43:57.831401 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 25795
I20260807 00:43:57.831542 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:43:57.831780 1490 external_mini_cluster.cc:1428] Setting key c0b7e81c6e3881e87fececd91cb15ee6
I20260807 00:43:57.832624 25886 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8421fa89d9481f8cb86aad80131a73" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35165 } }
I20260807 00:43:57.832849 25886 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:57.832886 25886 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a8421fa89d9481f8cb86aad80131a73, State: Initialized, Role: FOLLOWER
I20260807 00:43:57.833005 25886 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8421fa89d9481f8cb86aad80131a73" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35165 } }
I20260807 00:43:57.833091 25886 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:43:57.833128 25886 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:43:57.833185 25886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:57.834172 25886 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8421fa89d9481f8cb86aad80131a73" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35165 } }
I20260807 00:43:57.834285 25886 leader_election.cc:302] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a8421fa89d9481f8cb86aad80131a73; no voters:
I20260807 00:43:57.834486 25886 leader_election.cc:288] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:43:57.834607 25894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:43:57.834805 25886 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:43:57.834877 25894 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [term 1 LEADER]: Becoming Leader. State: Replica: 7a8421fa89d9481f8cb86aad80131a73, State: Running, Role: LEADER
I20260807 00:43:57.835079 25894 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8421fa89d9481f8cb86aad80131a73" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35165 } }
I20260807 00:43:57.835531 25894 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a8421fa89d9481f8cb86aad80131a73. Latest consensus state: current_term: 1 leader_uuid: "7a8421fa89d9481f8cb86aad80131a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8421fa89d9481f8cb86aad80131a73" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35165 } } }
I20260807 00:43:57.836036 25894 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:57.835932 25895 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a8421fa89d9481f8cb86aad80131a73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a8421fa89d9481f8cb86aad80131a73" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 35165 } } }
I20260807 00:43:57.836459 25895 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:57.836580 25908 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:43:57.837353 25908 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:43:57.839376 25908 catalog_manager.cc:1347] Generated new cluster ID: a3ad73b7bb5b494b969221feaf0b5917
I20260807 00:43:57.839460 25908 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:43:57 dist-test-slave-h6n0 krb5kdc[25778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:43:57.845705 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:57.832469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47634 (local address 127.1.116.190:35165)
0807 00:43:57.832752 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:57.832762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:57.832804 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:43:57.832934 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:57.832941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:57.832997 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:57.833177 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:57.833992 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:57.835598 (+ 1606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:57.836882 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:57.837439 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:57.837595 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:57.840881 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:57.840886 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:57.840887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:57.840902 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:57.843183 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:57.843607 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:57.843610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:57.843611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:57.843737 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:57.843953 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:57.843954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:57.843954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:57.844346 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:57.844540 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:57.845108 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 00:43:57.845280 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":142,"threads_started":1}
I20260807 00:43:57.848896 25908 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:43:57.849678 25908 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:43:57.862076 25908 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73: Generated new TSK 0
I20260807 00:43:57.862318 25908 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:43:57.893718 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35165
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:57.981874 25927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:57.982079 25927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:57.982102 25927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:57.983297 25927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:57.983345 25927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:57.983371 25927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:43:57.985016 25927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:57.985297 25927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:57.985606 25927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:57.990921 25935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:57.991088 25934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:57.990916 25937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:57.991477 25927 server_base.cc:1034] running on GCE node
I20260807 00:43:57.991708 25927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:57.991937 25927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:57.993139 25927 hybrid_clock.cc:648] HybridClock initialized: now 1786063437993115 us; error 62 us; skew 500 ppm
Aug 07 00:43:57 dist-test-slave-h6n0 krb5kdc[25778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063437, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:57.996070 25927 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:43:57.996523 25927 webserver.cc:460] Webserver started at http://127.1.116.129:46637/ using document root <none> and password file <none>
I20260807 00:43:57.996775 25927 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:57.996846 25927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:57.997008 25927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:57.998042 25927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "217f7f2045d440dba53d48b6eb6d1c36"
format_stamp: "Formatted at 2026-08-07 00:43:57 on dist-test-slave-h6n0"
server_key: "0780a5fa1c9507c86a42e1052487b58e"
server_key_iv: "7077629ece0220e4da0af3fd905c8457"
server_key_version: "encryptionkey@0"
I20260807 00:43:57.998405 25927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "217f7f2045d440dba53d48b6eb6d1c36"
format_stamp: "Formatted at 2026-08-07 00:43:57 on dist-test-slave-h6n0"
server_key: "0780a5fa1c9507c86a42e1052487b58e"
server_key_iv: "7077629ece0220e4da0af3fd905c8457"
server_key_version: "encryptionkey@0"
I20260807 00:43:57.999918 25927 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:43:58.000835 25945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.001088 25927 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:43:58.001192 25927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "217f7f2045d440dba53d48b6eb6d1c36"
format_stamp: "Formatted at 2026-08-07 00:43:57 on dist-test-slave-h6n0"
server_key: "0780a5fa1c9507c86a42e1052487b58e"
server_key_iv: "7077629ece0220e4da0af3fd905c8457"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.001279 25927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:58.009843 25927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:58.012205 25927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:58.012370 25927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:58.012607 25927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:58.012936 25927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:58.012995 25927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.013027 25927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:58.013056 25927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.020151 25927 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:36029
I20260807 00:43:58.020272 26118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:36029 every 8 connection(s)
I20260807 00:43:58.020563 25927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
Aug 07 00:43:58 dist-test-slave-h6n0 krb5kdc[25778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063437, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:43:58.030380 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.021419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:42039 (local address 127.1.116.190:35165)
0807 00:43:58.021468 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.021470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.021704 (+ 234us) server_negotiation.cc:407] Connection header received
0807 00:43:58.022244 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.022246 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.022263 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.022319 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:58.023252 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.023773 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.024765 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.024986 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.025068 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.027432 (+ 2364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:58.027433 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:58.027433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:58.027441 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:58.028926 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.029346 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.029348 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.029348 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.029419 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.029683 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.029684 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.029685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.029803 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:58.029877 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.030186 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.030284 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 00:43:58.030507 26123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.021380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35165 (local address 127.1.116.129:42039)
0807 00:43:58.021573 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.021612 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.022130 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.022385 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.022390 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.022972 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:58.023147 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.023152 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.023877 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.023878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.024665 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.024667 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.025079 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.025080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.025208 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.025700 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:58.025708 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:58.027316 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:58.029032 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:58.029040 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:58.029042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:58.029264 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:58.029494 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:58.029495 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:58.029495 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:58.029604 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:58.029917 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:58.029925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:58.030044 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.030147 (+ 103us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.030295 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
I20260807 00:43:58.030767 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 25927
I20260807 00:43:58.030896 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:43:58.031132 1490 external_mini_cluster.cc:1428] Setting key 2daa8fd036bf2de24068cb2f0ead9fa4
I20260807 00:43:58.031250 26119 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35165
I20260807 00:43:58.031352 26119 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:58.031546 26119 heartbeater.cc:511] Master 127.1.116.190:35165 requested a full tablet report, sending...
I20260807 00:43:58.032215 25836 ts_manager.cc:194] Registered new tserver with Master: 217f7f2045d440dba53d48b6eb6d1c36 (127.1.116.129:36029)
I20260807 00:43:58.032922 25836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:42039
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:43:58.060950 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35165
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:58.143667 26129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:58.143905 26129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:58.143944 26129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:58.145054 26129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:58.145128 26129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:58.145174 26129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:43:58.146723 26129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:58.146979 26129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:58.147225 26129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:58.152527 26136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.152555 26137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.152555 26139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:58.152835 26129 server_base.cc:1034] running on GCE node
I20260807 00:43:58.153067 26129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:58.153318 26129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:58.154487 26129 hybrid_clock.cc:648] HybridClock initialized: now 1786063438154480 us; error 44 us; skew 500 ppm
Aug 07 00:43:58 dist-test-slave-h6n0 krb5kdc[25778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063438, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:58.157513 26129 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:43:58.157984 26129 webserver.cc:460] Webserver started at http://127.1.116.130:34733/ using document root <none> and password file <none>
I20260807 00:43:58.158231 26129 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:58.158299 26129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:58.158456 26129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:58.159518 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "c441844d954d478a856ba20833c67fde"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "a84c6fff38276a0362b638c54414e7f4"
server_key_iv: "2c777fec3454661220a50044e2038aa9"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.159883 26129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "c441844d954d478a856ba20833c67fde"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "a84c6fff38276a0362b638c54414e7f4"
server_key_iv: "2c777fec3454661220a50044e2038aa9"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.161322 26129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:43:58.162122 26147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.162391 26129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:58.162518 26129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "c441844d954d478a856ba20833c67fde"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "a84c6fff38276a0362b638c54414e7f4"
server_key_iv: "2c777fec3454661220a50044e2038aa9"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.162650 26129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:58.177418 26129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:58.179708 26129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:58.179864 26129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:58.180104 26129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:58.180416 26129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:58.180473 26129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.180507 26129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:58.180533 26129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.187877 26129 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:33545
I20260807 00:43:58.187965 26311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:33545 every 8 connection(s)
I20260807 00:43:58.188302 26129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
Aug 07 00:43:58 dist-test-slave-h6n0 krb5kdc[25778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063438, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:43:58.197409 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 26129
I20260807 00:43:58.197559 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:43:58.197841 1490 external_mini_cluster.cc:1428] Setting key 826645d5120d4029489c12ef6e3ecdde
I20260807 00:43:58.198909 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.189207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:40839 (local address 127.1.116.190:35165)
0807 00:43:58.189249 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.189251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.189500 (+ 249us) server_negotiation.cc:407] Connection header received
0807 00:43:58.190032 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.190033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.190046 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.190102 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:58.191087 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.191622 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.192613 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.192825 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.192907 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.195300 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:58.195301 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:58.195302 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:58.195308 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:58.197007 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.197670 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.197672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.197673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.197765 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.198104 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.198106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.198106 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.198269 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:58.198358 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.198722 (+ 364us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.198814 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:43:58.199188 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.189170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35165 (local address 127.1.116.130:40839)
0807 00:43:58.189368 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.189413 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.189921 (+ 508us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.190176 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.190179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.190802 (+ 623us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:58.190988 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.190993 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.191724 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.191725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.192506 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.192508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.192915 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.192916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.193034 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.193530 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:58.193538 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:58.195168 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:58.197102 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:58.197109 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:58.197111 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:58.197517 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:58.197852 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:58.197853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:58.197854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:58.197998 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:58.198357 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:58.198366 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:58.198505 (+ 139us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.198695 (+ 190us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.198868 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":105,"threads_started":1}
I20260807 00:43:58.199762 26312 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35165
I20260807 00:43:58.199905 26312 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:58.200191 26312 heartbeater.cc:511] Master 127.1.116.190:35165 requested a full tablet report, sending...
I20260807 00:43:58.200625 25836 ts_manager.cc:194] Registered new tserver with Master: c441844d954d478a856ba20833c67fde (127.1.116.130:33545)
I20260807 00:43:58.201015 25836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:40839
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:43:58.231767 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35165
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:42005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:43:58.329936 26321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:58.330215 26321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:58.330242 26321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:58.331612 26321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:58.331672 26321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:58.331702 26321 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:43:58.333230 26321 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:42005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:35165
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:58.333458 26321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:58.333673 26321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:58.339712 26333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.339705 26330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.339712 26331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:58.340230 26321 server_base.cc:1034] running on GCE node
I20260807 00:43:58.340509 26321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:58.340835 26321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:58.342017 26321 hybrid_clock.cc:648] HybridClock initialized: now 1786063438341996 us; error 41 us; skew 500 ppm
Aug 07 00:43:58 dist-test-slave-h6n0 krb5kdc[25778](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063438, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:43:58.344987 26321 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:43:58.345526 26321 webserver.cc:460] Webserver started at http://127.1.116.131:32895/ using document root <none> and password file <none>
I20260807 00:43:58.345745 26321 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:58.345793 26321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:58.345889 26321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:58.347126 26321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "04196dd7d51f412287ad9448c77175a8"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "a51966d0cb04f8446e86e7b8c04ad3b5"
server_key_iv: "41113a43fd54a11eaf122be33116a488"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.347484 26321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "04196dd7d51f412287ad9448c77175a8"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "a51966d0cb04f8446e86e7b8c04ad3b5"
server_key_iv: "41113a43fd54a11eaf122be33116a488"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.349123 26321 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:43:58.350028 26341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.350325 26321 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:58.350421 26321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "04196dd7d51f412287ad9448c77175a8"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "a51966d0cb04f8446e86e7b8c04ad3b5"
server_key_iv: "41113a43fd54a11eaf122be33116a488"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.350484 26321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:58.385198 26321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:58.388012 26321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:58.388176 26321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:58.388453 26321 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:58.388877 26321 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:58.388944 26321 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.388985 26321 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:58.389029 26321 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.397336 26321 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44595
I20260807 00:43:58.397383 26506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44595 every 8 connection(s)
I20260807 00:43:58.397804 26321 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:43:58.399871 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 26321
I20260807 00:43:58.400014 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:43:58.400254 1490 external_mini_cluster.cc:1428] Setting key 8f334cfae12ed26e44accd92ea60f99f
Aug 07 00:43:58 dist-test-slave-h6n0 krb5kdc[25778](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063438, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:43:58.409487 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.398753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:38175 (local address 127.1.116.190:35165)
0807 00:43:58.398907 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.398909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.399108 (+ 199us) server_negotiation.cc:407] Connection header received
0807 00:43:58.399736 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.399738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.399755 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.399888 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:58.401427 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.402042 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.403291 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.403545 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.403703 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.406168 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:58.406170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:58.406170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:58.406178 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:58.407758 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.408311 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.408312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.408312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.408382 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.408675 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.408677 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.408677 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.408812 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:58.408885 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.409277 (+ 392us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.409384 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 00:43:58.409770 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.398749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:35165 (local address 127.1.116.131:38175)
0807 00:43:58.398940 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.398990 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.399589 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.400045 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.400051 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.401025 (+ 974us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:58.401311 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.401318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.402158 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.402159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.403145 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.403148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.403683 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.403684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.403817 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.404387 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:43:58.404397 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:58.406038 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:58.407870 (+ 1832us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:58.407878 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:58.407880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:58.408157 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:58.408461 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:43:58.408462 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:43:58.408462 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:43:58.408577 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:43:58.408925 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:58.408933 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:58.409091 (+ 158us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.409212 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.409456 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260807 00:43:58.410619 26507 heartbeater.cc:347] Connected to a master server at 127.1.116.190:35165
I20260807 00:43:58.410737 26507 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:58.410925 26507 heartbeater.cc:511] Master 127.1.116.190:35165 requested a full tablet report, sending...
I20260807 00:43:58.411342 25836 ts_manager.cc:194] Registered new tserver with Master: 04196dd7d51f412287ad9448c77175a8 (127.1.116.131:44595)
I20260807 00:43:58.411911 25836 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:38175
I20260807 00:43:58.412611 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:43:58.420840 25893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.414201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.1.116.190:35165)
0807 00:43:58.414238 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.414240 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.414460 (+ 220us) server_negotiation.cc:407] Connection header received
0807 00:43:58.414567 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.414568 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.414578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.414636 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:58.415339 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.415871 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.416611 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.416863 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.416943 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.418199 (+ 1256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:58.418201 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:58.418201 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:43:58.418208 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:58.419640 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.419993 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.419995 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.419995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.420051 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:43:58.420276 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:43:58.420277 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:43:58.420277 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:43:58.420423 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:58.420534 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.420665 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.420742 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:43:58.422755 25836 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:43:58.424413 25836 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:43:58.432052 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.428232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39782 (local address 127.1.116.130:33545)
0807 00:43:58.428437 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.428441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.428451 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:43:58.428490 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.428492 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.428506 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.428617 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:58.428784 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.429540 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.430889 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.431592 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.431676 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.431735 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.431924 (+ 189us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.431967 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":103,"threads_started":1}
I20260807 00:43:58.432093 26525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.427711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:33545 (local address 127.0.0.1:39782)
0807 00:43:58.428316 (+ 605us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.428330 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.428400 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.428634 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.428638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.428640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:58.428705 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.428707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.429651 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.429652 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.430778 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.430782 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.431698 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.431699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.431789 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.431805 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.431893 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.431964 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260807 00:43:58.432334 26522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.427967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44595 (local address 127.0.0.1:37592)
0807 00:43:58.428481 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.428492 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.428545 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.428868 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.428869 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.428872 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:58.428923 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.428925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.430264 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.430264 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.431242 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.431245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.432086 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.432087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.432170 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.432175 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.432210 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.432260 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"mutex_wait_us":61,"thread_start_us":130,"threads_started":1}
I20260807 00:43:58.433221 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.427918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.1.116.129:36029)
0807 00:43:58.428231 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.428235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.428250 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:43:58.428548 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.428553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.428574 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.428658 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:58.428928 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.430115 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.431711 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.432835 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.432952 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.433008 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.433101 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.433142 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":187,"threads_started":1}
I20260807 00:43:58.433215 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.427711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:36029 (local address 127.0.0.1:38268)
0807 00:43:58.427940 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.427989 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.428369 (+ 380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.428768 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.428769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.428772 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:58.428843 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.428846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.430254 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.430255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.431594 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.431597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.432948 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.432949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.433023 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.433027 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.433087 (+ 60us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.433140 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":64,"thread_start_us":101,"threads_started":1}
I20260807 00:43:58.433135 26214 tablet_service.cc:1467] Processing CreateTablet for tablet 71de7af4eac64af3a2bf245a4d5f874d (DEFAULT_TABLE table=test-table [id=ceb65ad135df45ccb7e6c2f0e64dae74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:58.433615 26214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71de7af4eac64af3a2bf245a4d5f874d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:58.435109 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.428125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37592 (local address 127.1.116.131:44595)
0807 00:43:58.428695 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.428700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.428711 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:43:58.428754 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.428756 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.428773 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.428864 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:58.428996 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.430130 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.431338 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.431989 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.434740 (+ 2751us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.434932 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.434969 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.435025 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":146,"threads_started":1}
I20260807 00:43:58.435488 26012 tablet_service.cc:1467] Processing CreateTablet for tablet 71de7af4eac64af3a2bf245a4d5f874d (DEFAULT_TABLE table=test-table [id=ceb65ad135df45ccb7e6c2f0e64dae74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:58.435915 26012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71de7af4eac64af3a2bf245a4d5f874d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:58.437413 26529 tablet_bootstrap.cc:492] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Bootstrap starting.
I20260807 00:43:58.437925 26416 tablet_service.cc:1467] Processing CreateTablet for tablet 71de7af4eac64af3a2bf245a4d5f874d (DEFAULT_TABLE table=test-table [id=ceb65ad135df45ccb7e6c2f0e64dae74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:58.438299 26416 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 71de7af4eac64af3a2bf245a4d5f874d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:58.438412 26529 tablet_bootstrap.cc:654] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:58.439427 26529 log.cc:824] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:58.439663 26530 tablet_bootstrap.cc:492] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36: Bootstrap starting.
I20260807 00:43:58.440550 26530 tablet_bootstrap.cc:654] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:58.440783 26529 tablet_bootstrap.cc:492] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: No bootstrap required, opened a new log
I20260807 00:43:58.440882 26529 ts_tablet_manager.cc:1397] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 00:43:58.441169 26530 log.cc:824] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:58.441952 26533 tablet_bootstrap.cc:492] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Bootstrap starting.
I20260807 00:43:58.442067 26530 tablet_bootstrap.cc:492] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36: No bootstrap required, opened a new log
I20260807 00:43:58.442335 26530 ts_tablet_manager.cc:1397] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:43:58.442858 26533 tablet_bootstrap.cc:654] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:58.443406 26533 log.cc:824] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:58.443341 26529 raft_consensus.cc:348] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.443553 26529 raft_consensus.cc:374] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:58.443638 26529 raft_consensus.cc:729] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c441844d954d478a856ba20833c67fde, State: Initialized, Role: FOLLOWER
I20260807 00:43:58.443805 26529 consensus_queue.cc:260] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.444164 26533 tablet_bootstrap.cc:492] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: No bootstrap required, opened a new log
I20260807 00:43:58.444265 26533 ts_tablet_manager.cc:1397] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 00:43:58.444303 26529 ts_tablet_manager.cc:1428] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:43:58.444610 26530 raft_consensus.cc:348] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.444725 26530 raft_consensus.cc:374] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:58.444743 26530 raft_consensus.cc:729] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 217f7f2045d440dba53d48b6eb6d1c36, State: Initialized, Role: FOLLOWER
I20260807 00:43:58.444820 26530 consensus_queue.cc:260] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.445029 26530 ts_tablet_manager.cc:1428] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:43:58.445152 26119 heartbeater.cc:503] Master 127.1.116.190:35165 was elected leader, sending a full tablet report...
I20260807 00:43:58.446096 26533 raft_consensus.cc:348] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.446449 26533 raft_consensus.cc:374] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:58.446588 26533 raft_consensus.cc:729] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04196dd7d51f412287ad9448c77175a8, State: Initialized, Role: FOLLOWER
I20260807 00:43:58.446619 26312 heartbeater.cc:503] Master 127.1.116.190:35165 was elected leader, sending a full tablet report...
I20260807 00:43:58.446724 26533 consensus_queue.cc:260] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.447052 26533 ts_tablet_manager.cc:1428] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:43:58.447261 26507 heartbeater.cc:503] Master 127.1.116.190:35165 was elected leader, sending a full tablet report...
I20260807 00:43:58.474455 26541 raft_consensus.cc:482] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:58.474695 26541 raft_consensus.cc:504] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.475170 26541 leader_election.cc:288] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 217f7f2045d440dba53d48b6eb6d1c36 (127.1.116.129:36029), c441844d954d478a856ba20833c67fde (127.1.116.130:33545)
I20260807 00:43:58.479506 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.475400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:35119 (local address 127.1.116.129:36029)
0807 00:43:58.475451 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.475454 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.475474 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:43:58.475605 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.475606 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.475618 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.475683 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:58.475939 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.476732 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.478446 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.479066 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.479152 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.479161 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.479390 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.479429 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:43:58.479502 26528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.475504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:58959 (local address 127.1.116.130:33545)
0807 00:43:58.475574 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.475576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.475775 (+ 199us) server_negotiation.cc:407] Connection header received
0807 00:43:58.475807 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.475807 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.475815 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.475857 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:58.476041 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.476731 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.478446 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.479070 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.479152 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.479161 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.479389 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.479429 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:43:58.479543 26509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.475333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:36029 (local address 127.1.116.131:35119)
0807 00:43:58.475412 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.475429 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.475491 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.475720 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.475722 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.475724 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:58.475831 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.475833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.476839 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.476840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.478338 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.478341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.479179 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.479181 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.479282 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.479284 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.479362 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.479415 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":159}
I20260807 00:43:58.479557 26543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.475412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:33545 (local address 127.1.116.131:58959)
0807 00:43:58.475633 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.475642 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.475733 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.475891 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.475892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.475894 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:58.475965 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.475966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.476839 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.476840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.478338 (+ 1498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.478341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.479205 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.479206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.479277 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.479284 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.479362 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.479415 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":100,"thread_start_us":162,"threads_started":1}
I20260807 00:43:58.480078 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de7af4eac64af3a2bf245a4d5f874d" candidate_uuid: "04196dd7d51f412287ad9448c77175a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c441844d954d478a856ba20833c67fde" is_pre_election: true
I20260807 00:43:58.480090 26045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de7af4eac64af3a2bf245a4d5f874d" candidate_uuid: "04196dd7d51f412287ad9448c77175a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "217f7f2045d440dba53d48b6eb6d1c36" is_pre_election: true
I20260807 00:43:58.480244 26249 raft_consensus.cc:2454] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04196dd7d51f412287ad9448c77175a8 in term 0.
I20260807 00:43:58.480298 26045 raft_consensus.cc:2454] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04196dd7d51f412287ad9448c77175a8 in term 0.
I20260807 00:43:58.480567 26344 leader_election.cc:302] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04196dd7d51f412287ad9448c77175a8, c441844d954d478a856ba20833c67fde; no voters:
I20260807 00:43:58.480724 26541 raft_consensus.cc:2781] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:43:58.480787 26541 raft_consensus.cc:482] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:43:58.480827 26541 raft_consensus.cc:3037] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:58.481634 26541 raft_consensus.cc:504] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.481824 26541 leader_election.cc:288] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [CANDIDATE]: Term 1 election: Requested vote from peers 217f7f2045d440dba53d48b6eb6d1c36 (127.1.116.129:36029), c441844d954d478a856ba20833c67fde (127.1.116.130:33545)
I20260807 00:43:58.481978 26045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de7af4eac64af3a2bf245a4d5f874d" candidate_uuid: "04196dd7d51f412287ad9448c77175a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "217f7f2045d440dba53d48b6eb6d1c36"
I20260807 00:43:58.482021 26249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "71de7af4eac64af3a2bf245a4d5f874d" candidate_uuid: "04196dd7d51f412287ad9448c77175a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c441844d954d478a856ba20833c67fde"
I20260807 00:43:58.482074 26045 raft_consensus.cc:3037] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:58.482120 26249 raft_consensus.cc:3037] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:58.482937 26249 raft_consensus.cc:2454] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04196dd7d51f412287ad9448c77175a8 in term 1.
I20260807 00:43:58.483114 26045 raft_consensus.cc:2454] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04196dd7d51f412287ad9448c77175a8 in term 1.
I20260807 00:43:58.483208 26344 leader_election.cc:302] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04196dd7d51f412287ad9448c77175a8, c441844d954d478a856ba20833c67fde; no voters:
I20260807 00:43:58.483315 26541 raft_consensus.cc:2781] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:43:58.483538 26541 raft_consensus.cc:686] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 1 LEADER]: Becoming Leader. State: Replica: 04196dd7d51f412287ad9448c77175a8, State: Running, Role: LEADER
I20260807 00:43:58.483649 26541 consensus_queue.cc:237] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } }
I20260807 00:43:58.484589 25836 catalog_manager.cc:5495] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 04196dd7d51f412287ad9448c77175a8 (127.1.116.131). New cstate: current_term: 1 leader_uuid: "04196dd7d51f412287ad9448c77175a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "04196dd7d51f412287ad9448c77175a8" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44595 } health_report { overall_health: HEALTHY } } }
I20260807 00:43:58.493474 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.491161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.1.116.131:44595)
0807 00:43:58.491240 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.491242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.491250 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:43:58.491287 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.491288 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.491303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.491380 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:43:58.491489 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.491951 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.492696 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.492901 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.492975 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.493137 (+ 162us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:43:58.493234 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:43:58.493318 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.493369 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.493390 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:43:58.495645 26045 raft_consensus.cc:1261] T 71de7af4eac64af3a2bf245a4d5f874d P 217f7f2045d440dba53d48b6eb6d1c36 [term 1 FOLLOWER]: Refusing update from remote peer 04196dd7d51f412287ad9448c77175a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:43:58.495652 26249 raft_consensus.cc:1261] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 1 FOLLOWER]: Refusing update from remote peer 04196dd7d51f412287ad9448c77175a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:43:58.496013 26541 consensus_queue.cc:1035] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "217f7f2045d440dba53d48b6eb6d1c36" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 36029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:43:58.496196 26544 consensus_queue.cc:1035] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c441844d954d478a856ba20833c67fde" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 33545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:43:58.501544 25835 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:43:58.501658 25835 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47640:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:43:58.502583 25835 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a8421fa89d9481f8cb86aad80131a73: Sending DeleteTablet for 3 replicas of tablet 71de7af4eac64af3a2bf245a4d5f874d
I20260807 00:43:58.502985 26214 tablet_service.cc:1514] Processing DeleteTablet for tablet 71de7af4eac64af3a2bf245a4d5f874d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:58 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:39782
I20260807 00:43:58.503032 26012 tablet_service.cc:1514] Processing DeleteTablet for tablet 71de7af4eac64af3a2bf245a4d5f874d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:58 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:38268
I20260807 00:43:58.503326 26565 tablet_replica.cc:330] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: stopping tablet replica
I20260807 00:43:58.503427 26416 tablet_service.cc:1514] Processing DeleteTablet for tablet 71de7af4eac64af3a2bf245a4d5f874d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:58 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:37592
I20260807 00:43:58.503489 26565 raft_consensus.cc:2229] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:43:58.503608 26565 raft_consensus.cc:2258] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:58.503660 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 25927
I20260807 00:43:58.504132 26565 ts_tablet_manager.cc:1905] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:43:58.506569 26565 ts_tablet_manager.cc:1918] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:43:58.506660 26565 log.cc:1196] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/71de7af4eac64af3a2bf245a4d5f874d
I20260807 00:43:58.506974 26565 ts_tablet_manager.cc:1939] T 71de7af4eac64af3a2bf245a4d5f874d P c441844d954d478a856ba20833c67fde: Deleting consensus metadata
I20260807 00:43:58.512624 25819 catalog_manager.cc:4841] TS c441844d954d478a856ba20833c67fde (127.1.116.130:33545): tablet 71de7af4eac64af3a2bf245a4d5f874d (table test-table [id=ceb65ad135df45ccb7e6c2f0e64dae74]) successfully deleted
I20260807 00:43:58.514668 26566 tablet_replica.cc:330] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: stopping tablet replica
I20260807 00:43:58.514792 26566 raft_consensus.cc:2229] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:43:58.514943 26566 raft_consensus.cc:2258] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:58.515528 26566 ts_tablet_manager.cc:1905] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:43:58.517328 26566 ts_tablet_manager.cc:1918] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:43:58.517463 26566 log.cc:1196] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/71de7af4eac64af3a2bf245a4d5f874d
I20260807 00:43:58.517867 26566 ts_tablet_manager.cc:1939] T 71de7af4eac64af3a2bf245a4d5f874d P 04196dd7d51f412287ad9448c77175a8: Deleting consensus metadata
W20260807 00:43:58.518834 25820 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:36029 (error 108)
I20260807 00:43:58.519011 25819 catalog_manager.cc:4841] TS 04196dd7d51f412287ad9448c77175a8 (127.1.116.131:44595): tablet 71de7af4eac64af3a2bf245a4d5f874d (table test-table [id=ceb65ad135df45ccb7e6c2f0e64dae74]) successfully deleted
I20260807 00:43:58.519259 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 26129
I20260807 00:43:58.519511 26524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.519376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:36029 (local address 127.0.0.1:38274)
0807 00:43:58.519441 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.519476 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:36029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 00:43:58.519670 25820 catalog_manager.cc:4584] TS 217f7f2045d440dba53d48b6eb6d1c36 (127.1.116.129:36029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 71de7af4eac64af3a2bf245a4d5f874d: Network error: Client connection negotiation failed: client connection to 127.1.116.129:36029: connect: Connection refused (error 111)
I20260807 00:43:58.530356 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 26321
I20260807 00:43:58.540832 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 25795
2026-08-07T00:43:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2249 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T00:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:43:58Z Disabled control of system clock
I20260807 00:43:58.575791 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:33485
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:33485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:43:58.671906 26571 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:58.672179 26571 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:58.672204 26571 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:58.673372 26571 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:43:58.673409 26571 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:58.673422 26571 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:58.673435 26571 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:43:58.673447 26571 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:43:58.674893 26571 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:33485
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:33485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26571
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:58.675153 26571 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:58.675448 26571 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:58.681052 26583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.681030 26578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.681030 26580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:58.681082 26571 server_base.cc:1034] running on GCE node
I20260807 00:43:58.681665 26571 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:58.682025 26571 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:58.683218 26571 hybrid_clock.cc:648] HybridClock initialized: now 1786063438683174 us; error 59 us; skew 500 ppm
I20260807 00:43:58.684803 26571 webserver.cc:460] Webserver started at http://127.1.116.190:38149/ using document root <none> and password file <none>
I20260807 00:43:58.685127 26571 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:58.685189 26571 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:58.685362 26571 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:58.686734 26571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "50029f6a15f8f4c853e1e49e20a896f9"
server_key_iv: "162d28b3578254804eeff99963c54276"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.687142 26571 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "50029f6a15f8f4c853e1e49e20a896f9"
server_key_iv: "162d28b3578254804eeff99963c54276"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.688690 26571 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:43:58.689632 26592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.689869 26571 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:43:58.689972 26571 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "50029f6a15f8f4c853e1e49e20a896f9"
server_key_iv: "162d28b3578254804eeff99963c54276"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.690062 26571 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:58.702736 26571 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:58.703099 26571 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:58.703264 26571 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:58.708416 26571 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:33485
I20260807 00:43:58.708498 26669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:33485 every 8 connection(s)
I20260807 00:43:58.708887 26571 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:43:58.710003 26670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:58.711030 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 26571
I20260807 00:43:58.711164 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:43:58.711423 1490 external_mini_cluster.cc:1428] Setting key 7a28b5403fd2dee279cbceb40a82bcd3
I20260807 00:43:58.713172 26670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0: Bootstrap starting.
I20260807 00:43:58.713907 26670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:58.714238 26670 log.cc:824] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:58.715175 26670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0: No bootstrap required, opened a new log
I20260807 00:43:58.717707 26670 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33485 } }
I20260807 00:43:58.717890 26670 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:58.717939 26670 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcdeccdb8f484beb9e5fa7fcd7bddab0, State: Initialized, Role: FOLLOWER
I20260807 00:43:58.718155 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.712040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51666 (local address 127.1.116.190:33485)
0807 00:43:58.712310 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.712320 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.712356 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:43:58.712870 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.712879 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.712961 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.713125 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:58.713392 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.714771 (+ 1379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.715883 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.716513 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.716651 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.717439 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:58.717443 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:58.717443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:43:58.717651 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:58.717698 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:43:58.717712 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:58.717789 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.717872 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.717954 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":106,"threads_started":1}
I20260807 00:43:58.718133 26670 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33485 } }
I20260807 00:43:58.718216 26670 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:43:58.718251 26670 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:43:58.718293 26670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:58.718845 26670 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33485 } }
I20260807 00:43:58.718991 26670 leader_election.cc:302] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcdeccdb8f484beb9e5fa7fcd7bddab0; no voters:
I20260807 00:43:58.719179 26670 leader_election.cc:288] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:43:58.719275 26677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:43:58.719551 26677 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [term 1 LEADER]: Becoming Leader. State: Replica: bcdeccdb8f484beb9e5fa7fcd7bddab0, State: Running, Role: LEADER
I20260807 00:43:58.719580 26670 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:43:58.719691 26677 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33485 } }
I20260807 00:43:58.720176 26678 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33485 } } }
I20260807 00:43:58.720208 26679 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcdeccdb8f484beb9e5fa7fcd7bddab0. Latest consensus state: current_term: 1 leader_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcdeccdb8f484beb9e5fa7fcd7bddab0" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 33485 } } }
I20260807 00:43:58.720316 26678 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:58.720326 26679 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:43:58.720585 26687 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:43:58.721150 26687 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:43:58.723256 26687 catalog_manager.cc:1347] Generated new cluster ID: 83095f0e97a24c628179f54a5c699000
I20260807 00:43:58.723337 26687 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:43:58.752561 26687 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:43:58.753176 26687 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:43:58.765833 26687 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0: Generated new TSK 0
I20260807 00:43:58.766016 26687 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:43:58.777206 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33485
--builtin_ntp_servers=127.1.116.148:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:43:58.881770 26699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:58.881966 26699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:58.881987 26699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:58.883170 26699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:58.883217 26699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:58.883241 26699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:43:58.884588 26699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:58.884831 26699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:58.885176 26699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:58.890275 26706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.890341 26707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:58.890430 26712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:58.890612 26699 server_base.cc:1034] running on GCE node
I20260807 00:43:58.890900 26699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:58.891146 26699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:58.892331 26699 hybrid_clock.cc:648] HybridClock initialized: now 1786063438892310 us; error 37 us; skew 500 ppm
I20260807 00:43:58.894006 26699 webserver.cc:460] Webserver started at http://127.1.116.129:38677/ using document root <none> and password file <none>
I20260807 00:43:58.894429 26699 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:58.894522 26699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:58.894709 26699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:58.895992 26699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "492738822aa643c491d5f7a5c205de65"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "ec4a82b95b7ce9390ac64ef9f19f6f32"
server_key_iv: "98d122dd032b271aacfe803597cb623e"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.896425 26699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "492738822aa643c491d5f7a5c205de65"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "ec4a82b95b7ce9390ac64ef9f19f6f32"
server_key_iv: "98d122dd032b271aacfe803597cb623e"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.897967 26699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 00:43:58.899019 26720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.899479 26699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:43:58.899650 26699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "492738822aa643c491d5f7a5c205de65"
format_stamp: "Formatted at 2026-08-07 00:43:58 on dist-test-slave-h6n0"
server_key: "ec4a82b95b7ce9390ac64ef9f19f6f32"
server_key_iv: "98d122dd032b271aacfe803597cb623e"
server_key_version: "encryptionkey@0"
I20260807 00:43:58.899756 26699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:58.914652 26699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:58.915055 26699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:58.915272 26699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:58.915575 26699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:58.915989 26699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:58.916064 26699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.916123 26699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:58.916152 26699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:58.924001 26699 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:35281
I20260807 00:43:58.924063 26888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:35281 every 8 connection(s)
I20260807 00:43:58.924482 26699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:43:58.925078 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 26699
I20260807 00:43:58.925190 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:43:58.925484 1490 external_mini_cluster.cc:1428] Setting key c660a8937156c31320ec64d3dbb54518
I20260807 00:43:58.926733 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33485
--builtin_ntp_servers=127.1.116.148:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:43:58.931767 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.925500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:46071 (local address 127.1.116.190:33485)
0807 00:43:58.925566 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:58.925568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:58.925913 (+ 345us) server_negotiation.cc:407] Connection header received
0807 00:43:58.926754 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:58.926757 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:58.926772 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:58.926847 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:58.928057 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.928570 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.929861 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.930223 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.930303 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.931410 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:58.931411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:58.931412 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:43:58.931417 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:58.931435 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:43:58.931440 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:58.931507 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:58.931615 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 00:43:58.931643 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 00:43:58.931921 26891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:58.925500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33485 (local address 127.1.116.129:46071)
0807 00:43:58.925751 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:58.925788 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:58.926616 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:58.927024 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:58.927030 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:58.927042 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:58.927938 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.927946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.928684 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.928686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.929751 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:58.929753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:58.930334 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:58.930335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:58.930463 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:58.931239 (+ 776us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:43:58.931240 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:58.931272 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:43:58.931301 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:58.931522 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:58.931527 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:58.931530 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:43:58.931592 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:43:58.931691 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":128,"threads_started":1}
I20260807 00:43:58.932570 26889 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33485
I20260807 00:43:58.932698 26889 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:58.932904 26889 heartbeater.cc:511] Master 127.1.116.190:33485 requested a full tablet report, sending...
I20260807 00:43:58.933543 26621 ts_manager.cc:194] Registered new tserver with Master: 492738822aa643c491d5f7a5c205de65 (127.1.116.129:35281)
I20260807 00:43:58.934595 26621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.129:46071
W20260807 00:43:59.010413 26892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:59.010637 26892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:59.010659 26892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:59.011721 26892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:59.011760 26892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:59.011785 26892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:43:59.013033 26892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:59.013218 26892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:59.013525 26892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:59.018963 26900 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:59.018987 26901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:59.019007 26892 server_base.cc:1034] running on GCE node
W20260807 00:43:59.019045 26905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:59.019459 26892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:59.019811 26892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:59.020970 26892 hybrid_clock.cc:648] HybridClock initialized: now 1786063439020950 us; error 36 us; skew 500 ppm
I20260807 00:43:59.022452 26892 webserver.cc:460] Webserver started at http://127.1.116.130:35917/ using document root <none> and password file <none>
I20260807 00:43:59.022826 26892 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:59.022889 26892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:59.023104 26892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:59.024505 26892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "f3524189c2254da0b804f7b4242fc8e1"
format_stamp: "Formatted at 2026-08-07 00:43:59 on dist-test-slave-h6n0"
server_key: "c037d4191a9acb9cf74b483d14a331ac"
server_key_iv: "2f336bbeb86c62f5a87d34b7bb8a3436"
server_key_version: "encryptionkey@0"
I20260807 00:43:59.025010 26892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "f3524189c2254da0b804f7b4242fc8e1"
format_stamp: "Formatted at 2026-08-07 00:43:59 on dist-test-slave-h6n0"
server_key: "c037d4191a9acb9cf74b483d14a331ac"
server_key_iv: "2f336bbeb86c62f5a87d34b7bb8a3436"
server_key_version: "encryptionkey@0"
I20260807 00:43:59.026681 26892 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260807 00:43:59.027660 26911 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:59.027920 26892 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:59.028048 26892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "f3524189c2254da0b804f7b4242fc8e1"
format_stamp: "Formatted at 2026-08-07 00:43:59 on dist-test-slave-h6n0"
server_key: "c037d4191a9acb9cf74b483d14a331ac"
server_key_iv: "2f336bbeb86c62f5a87d34b7bb8a3436"
server_key_version: "encryptionkey@0"
I20260807 00:43:59.028149 26892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:59.037058 26892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:59.037400 26892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:59.037562 26892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:59.037822 26892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:59.038180 26892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:59.038239 26892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:59.038283 26892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:59.038311 26892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:59.046082 26892 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:43665
I20260807 00:43:59.046135 27093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:43665 every 8 connection(s)
I20260807 00:43:59.046655 26892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:43:59.052526 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.047546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:50539 (local address 127.1.116.190:33485)
0807 00:43:59.047643 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.047645 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.047858 (+ 213us) server_negotiation.cc:407] Connection header received
0807 00:43:59.048450 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.048451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.048464 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.048534 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:59.049375 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.049922 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.051117 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.051311 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.051387 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.052261 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:59.052262 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:59.052263 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:43:59.052267 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:59.052282 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:43:59.052286 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:59.052337 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.052443 (+ 106us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.052465 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:43:59.052780 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.047476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33485 (local address 127.1.116.130:50539)
0807 00:43:59.047711 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.047757 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:59.048347 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:59.048627 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:59.048631 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:59.048641 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:59.049215 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.049219 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.050028 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.050029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.051007 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.051010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.051407 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.051407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.051528 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.052106 (+ 578us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:43:59.052106 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:59.052137 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:43:59.052163 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:59.052353 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:59.052358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:59.052360 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:43:59.052422 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:43:59.052511 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":115,"threads_started":1}
I20260807 00:43:59.053474 27096 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33485
I20260807 00:43:59.053573 27096 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:59.053748 27096 heartbeater.cc:511] Master 127.1.116.190:33485 requested a full tablet report, sending...
I20260807 00:43:59.054168 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 26892
I20260807 00:43:59.054200 26621 ts_manager.cc:194] Registered new tserver with Master: f3524189c2254da0b804f7b4242fc8e1 (127.1.116.130:43665)
I20260807 00:43:59.054323 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:43:59.054633 1490 external_mini_cluster.cc:1428] Setting key ea1dfe3330b0e1b6dd6162173e891b86
I20260807 00:43:59.054718 26621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.130:50539
I20260807 00:43:59.055840 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33485
--builtin_ntp_servers=127.1.116.148:45523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:43:59.148541 27103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:43:59.148797 27103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:43:59.148820 27103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:43:59.149951 27103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:43:59.149989 27103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:43:59.150014 27103 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:43:59.151451 27103 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:33485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:43:59.151685 27103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:43:59.151973 27103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:43:59.157775 27111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:59.157740 27119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:43:59.157748 27112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:43:59.158196 27103 server_base.cc:1034] running on GCE node
I20260807 00:43:59.158455 27103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:43:59.158788 27103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:43:59.159945 27103 hybrid_clock.cc:648] HybridClock initialized: now 1786063439159920 us; error 48 us; skew 500 ppm
I20260807 00:43:59.161435 27103 webserver.cc:460] Webserver started at http://127.1.116.131:34629/ using document root <none> and password file <none>
I20260807 00:43:59.161722 27103 fs_manager.cc:359] Metadata directory not provided
I20260807 00:43:59.161774 27103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:43:59.161962 27103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:43:59.163282 27103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "713002bf55bf4d04a01ed94ed4f53c87"
format_stamp: "Formatted at 2026-08-07 00:43:59 on dist-test-slave-h6n0"
server_key: "b0d428fe6b442cb8be0cba0683ff32de"
server_key_iv: "cb7709c4f81565278ecddfc225d2baa9"
server_key_version: "encryptionkey@0"
I20260807 00:43:59.163691 27103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "713002bf55bf4d04a01ed94ed4f53c87"
format_stamp: "Formatted at 2026-08-07 00:43:59 on dist-test-slave-h6n0"
server_key: "b0d428fe6b442cb8be0cba0683ff32de"
server_key_iv: "cb7709c4f81565278ecddfc225d2baa9"
server_key_version: "encryptionkey@0"
I20260807 00:43:59.165238 27103 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:43:59.166208 27127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:59.166474 27103 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:43:59.166632 27103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "713002bf55bf4d04a01ed94ed4f53c87"
format_stamp: "Formatted at 2026-08-07 00:43:59 on dist-test-slave-h6n0"
server_key: "b0d428fe6b442cb8be0cba0683ff32de"
server_key_iv: "cb7709c4f81565278ecddfc225d2baa9"
server_key_version: "encryptionkey@0"
I20260807 00:43:59.166728 27103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:43:59.184224 27103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:43:59.184657 27103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:43:59.184844 27103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:43:59.185129 27103 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:43:59.185560 27103 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:43:59.185628 27103 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:59.185670 27103 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:43:59.185698 27103 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:43:59.193444 27103 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:44501
I20260807 00:43:59.193535 27298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:44501 every 8 connection(s)
I20260807 00:43:59.193948 27103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:43:59.199723 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.194878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:56245 (local address 127.1.116.190:33485)
0807 00:43:59.194962 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.194964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.195238 (+ 274us) server_negotiation.cc:407] Connection header received
0807 00:43:59.195773 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.195774 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.195786 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.195872 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:59.196606 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.197191 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.198206 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.198394 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.198452 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.199464 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:59.199466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:59.199466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:43:59.199471 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:59.199486 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:43:59.199490 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:59.199536 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.199615 (+ 79us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.199639 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:43:59.199889 27302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.194858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:33485 (local address 127.1.116.131:56245)
0807 00:43:59.195095 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.195140 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:59.195672 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:59.195930 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:59.195933 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:59.195941 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:43:59.196494 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.196498 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.197301 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.197301 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.198120 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.198122 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.198498 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.198499 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.198703 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.199307 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:43:59.199308 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:43:59.199337 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:43:59.199365 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:43:59.199546 (+ 181us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:43:59.199548 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:43:59.199550 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:43:59.199600 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 00:43:59.199677 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
I20260807 00:43:59.200489 27299 heartbeater.cc:347] Connected to a master server at 127.1.116.190:33485
I20260807 00:43:59.200630 27299 heartbeater.cc:464] Registering TS with master...
I20260807 00:43:59.200881 27299 heartbeater.cc:511] Master 127.1.116.190:33485 requested a full tablet report, sending...
I20260807 00:43:59.201395 26621 ts_manager.cc:194] Registered new tserver with Master: 713002bf55bf4d04a01ed94ed4f53c87 (127.1.116.131:44501)
I20260807 00:43:59.201810 26621 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.131:56245
I20260807 00:43:59.203219 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 27103
I20260807 00:43:59.203337 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:43:59.203555 1490 external_mini_cluster.cc:1428] Setting key 9afe02d4416e06929426902ca9d518f4
I20260807 00:43:59.204690 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:43:59.209425 26675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.206269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.1.116.190:33485)
0807 00:43:59.206343 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.206345 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.206570 (+ 225us) server_negotiation.cc:407] Connection header received
0807 00:43:59.206657 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.206658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.206670 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.206739 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:43:59.206963 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.207464 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.208259 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.208438 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.208493 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.209176 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:43:59.209177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:43:59.209177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:43:59.209181 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:43:59.209193 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:43:59.209196 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:43:59.209240 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.209304 (+ 64us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.209322 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:43:59.211275 26621 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:43:59.213174 26621 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:43:59.220990 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.216312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60106 (local address 127.1.116.130:43665)
0807 00:43:59.216485 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.216491 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.216621 (+ 130us) server_negotiation.cc:407] Connection header received
0807 00:43:59.216695 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.216698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.216717 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.217194 (+ 477us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:59.217242 (+ 48us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.218372 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.219688 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.220437 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.220573 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.220676 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.220848 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.220896 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260807 00:43:59.221015 27312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.215975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44501 (local address 127.0.0.1:57624)
0807 00:43:59.216137 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.216184 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:59.216267 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:59.217040 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:59.217041 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:59.217044 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:59.217127 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.217130 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.218055 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.218056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.219359 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.219362 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.220626 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.220627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.220715 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.220723 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 00:43:59.220771 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 00:43:59.220843 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":100,"threads_started":1}
I20260807 00:43:59.221195 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.216412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.1.116.129:35281)
0807 00:43:59.216618 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.216624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.216886 (+ 262us) server_negotiation.cc:407] Connection header received
0807 00:43:59.216945 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.216947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.216964 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.217047 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:59.217368 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.218640 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.219902 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.220604 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.220676 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.220784 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.221061 (+ 277us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.221101 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":133,"threads_started":1}
I20260807 00:43:59.221442 27322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.216613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35281 (local address 127.0.0.1:41568)
0807 00:43:59.216731 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.216747 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:59.216809 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:59.217236 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:59.217237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:59.217239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:59.217277 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.217279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.218742 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.218743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.219797 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.219800 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.220818 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.220819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.220871 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.220874 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:43:59.220923 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 00:43:59.220969 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":2,"thread_start_us":58,"threads_started":1}
I20260807 00:43:59.221472 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.216106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57624 (local address 127.1.116.131:44501)
0807 00:43:59.216822 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.216827 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.216839 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:43:59.216878 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.216880 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.216897 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.217029 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:59.217222 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.217958 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.219462 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.220510 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.221152 (+ 642us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.221262 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.221293 (+ 31us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.221343 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":193,"threads_started":1}
I20260807 00:43:59.221627 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.216058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:43665 (local address 127.0.0.1:60106)
0807 00:43:59.216497 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.216507 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:59.216567 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:59.217027 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:59.217030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:59.217034 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:59.217113 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.217116 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.218478 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.218479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.219585 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.219587 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.220617 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.220618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.220706 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.220720 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 00:43:59.220829 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 00:43:59.220916 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":62,"threads_started":1}
I20260807 00:43:59.222368 26801 tablet_service.cc:1467] Processing CreateTablet for tablet 00d5e5dcc33346c5b955d03ef2df32d5 (DEFAULT_TABLE table=test-table [id=9c071eefd9294332bfaef9f2d144225c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:59.222419 27195 tablet_service.cc:1467] Processing CreateTablet for tablet 00d5e5dcc33346c5b955d03ef2df32d5 (DEFAULT_TABLE table=test-table [id=9c071eefd9294332bfaef9f2d144225c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:59.222807 26801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d5e5dcc33346c5b955d03ef2df32d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:59.222918 27195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d5e5dcc33346c5b955d03ef2df32d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:59.222888 26994 tablet_service.cc:1467] Processing CreateTablet for tablet 00d5e5dcc33346c5b955d03ef2df32d5 (DEFAULT_TABLE table=test-table [id=9c071eefd9294332bfaef9f2d144225c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:43:59.223265 26994 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00d5e5dcc33346c5b955d03ef2df32d5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:43:59.225574 27324 tablet_bootstrap.cc:492] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65: Bootstrap starting.
I20260807 00:43:59.226190 27324 tablet_bootstrap.cc:654] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:59.226503 27324 log.cc:824] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:59.227121 27327 tablet_bootstrap.cc:492] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: Bootstrap starting.
I20260807 00:43:59.227609 27324 tablet_bootstrap.cc:492] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65: No bootstrap required, opened a new log
I20260807 00:43:59.227670 27324 ts_tablet_manager.cc:1397] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:43:59.227839 27327 tablet_bootstrap.cc:654] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:59.228176 27327 log.cc:824] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:59.228511 27325 tablet_bootstrap.cc:492] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Bootstrap starting.
I20260807 00:43:59.229051 27327 tablet_bootstrap.cc:492] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: No bootstrap required, opened a new log
I20260807 00:43:59.229086 27325 tablet_bootstrap.cc:654] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Neither blocks nor log segments found. Creating new log.
I20260807 00:43:59.229147 27327 ts_tablet_manager.cc:1397] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:43:59.229404 27325 log.cc:824] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Log is configured to *not* fsync() on all Append() calls
I20260807 00:43:59.230085 27325 tablet_bootstrap.cc:492] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: No bootstrap required, opened a new log
I20260807 00:43:59.230188 27325 ts_tablet_manager.cc:1397] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:43:59.230372 27324 raft_consensus.cc:348] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.230574 27324 raft_consensus.cc:374] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:59.230612 27324 raft_consensus.cc:729] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 492738822aa643c491d5f7a5c205de65, State: Initialized, Role: FOLLOWER
I20260807 00:43:59.230746 27324 consensus_queue.cc:260] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.231022 27324 ts_tablet_manager.cc:1428] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 00:43:59.231194 26889 heartbeater.cc:503] Master 127.1.116.190:33485 was elected leader, sending a full tablet report...
I20260807 00:43:59.232079 27327 raft_consensus.cc:348] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.232239 27327 raft_consensus.cc:374] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:59.232265 27327 raft_consensus.cc:729] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3524189c2254da0b804f7b4242fc8e1, State: Initialized, Role: FOLLOWER
I20260807 00:43:59.232400 27327 consensus_queue.cc:260] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.232679 27327 ts_tablet_manager.cc:1428] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 00:43:59.232854 27096 heartbeater.cc:503] Master 127.1.116.190:33485 was elected leader, sending a full tablet report...
I20260807 00:43:59.233606 27325 raft_consensus.cc:348] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.233784 27325 raft_consensus.cc:374] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:43:59.233827 27325 raft_consensus.cc:729] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713002bf55bf4d04a01ed94ed4f53c87, State: Initialized, Role: FOLLOWER
I20260807 00:43:59.233958 27325 consensus_queue.cc:260] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.234210 27325 ts_tablet_manager.cc:1428] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260807 00:43:59.234297 27299 heartbeater.cc:503] Master 127.1.116.190:33485 was elected leader, sending a full tablet report...
I20260807 00:43:59.239701 27333 raft_consensus.cc:482] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:43:59.239809 27333 raft_consensus.cc:504] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.240146 27333 leader_election.cc:288] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 713002bf55bf4d04a01ed94ed4f53c87 (127.1.116.131:44501), 492738822aa643c491d5f7a5c205de65 (127.1.116.129:35281)
I20260807 00:43:59.244230 27321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.240831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:45489 (local address 127.1.116.129:35281)
0807 00:43:59.240895 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.240898 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.240907 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:43:59.240930 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.240930 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.240943 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.241005 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:59.241201 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.241812 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.243267 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.243818 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.243899 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.243908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.244116 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.244158 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 00:43:59.244230 27317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.240369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:57725 (local address 127.1.116.131:44501)
0807 00:43:59.240429 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.240432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.240447 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:43:59.240580 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.240582 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.240598 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.240675 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:43:59.240828 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.241542 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.243099 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.243812 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.243899 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.243908 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.244115 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.244158 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 00:43:59.244244 27099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.240289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:44501 (local address 127.1.116.130:57725)
0807 00:43:59.240355 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.240370 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:59.240422 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:59.240695 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:59.240697 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:59.240699 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:59.240769 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.240771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.241660 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.241663 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.242968 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.242971 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.243923 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.243924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.244013 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.244018 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:43:59.244089 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:43:59.244148 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":4}
I20260807 00:43:59.244249 27336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.240410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35281 (local address 127.1.116.130:45489)
0807 00:43:59.240693 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.240705 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:43:59.240811 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:43:59.241025 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:43:59.241026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:43:59.241028 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:43:59.241116 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.241117 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.241905 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.241905 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.243071 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:43:59.243073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.243923 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.243924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:43:59.244016 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.244022 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:43:59.244089 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:43:59.244145 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"mutex_wait_us":9,"thread_start_us":193,"threads_started":1}
I20260807 00:43:59.244685 27225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d5e5dcc33346c5b955d03ef2df32d5" candidate_uuid: "f3524189c2254da0b804f7b4242fc8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713002bf55bf4d04a01ed94ed4f53c87" is_pre_election: true
I20260807 00:43:59.244867 27225 raft_consensus.cc:2454] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3524189c2254da0b804f7b4242fc8e1 in term 0.
I20260807 00:43:59.244967 26828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d5e5dcc33346c5b955d03ef2df32d5" candidate_uuid: "f3524189c2254da0b804f7b4242fc8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492738822aa643c491d5f7a5c205de65" is_pre_election: true
I20260807 00:43:59.245113 26828 raft_consensus.cc:2454] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f3524189c2254da0b804f7b4242fc8e1 in term 0.
I20260807 00:43:59.245142 26918 leader_election.cc:302] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713002bf55bf4d04a01ed94ed4f53c87, f3524189c2254da0b804f7b4242fc8e1; no voters:
I20260807 00:43:59.245316 27333 raft_consensus.cc:2781] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:43:59.245406 27333 raft_consensus.cc:482] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:43:59.245438 27333 raft_consensus.cc:3037] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:59.246269 27333 raft_consensus.cc:504] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.246570 27333 leader_election.cc:288] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [CANDIDATE]: Term 1 election: Requested vote from peers 713002bf55bf4d04a01ed94ed4f53c87 (127.1.116.131:44501), 492738822aa643c491d5f7a5c205de65 (127.1.116.129:35281)
I20260807 00:43:59.246753 26828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d5e5dcc33346c5b955d03ef2df32d5" candidate_uuid: "f3524189c2254da0b804f7b4242fc8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "492738822aa643c491d5f7a5c205de65"
I20260807 00:43:59.246809 27225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00d5e5dcc33346c5b955d03ef2df32d5" candidate_uuid: "f3524189c2254da0b804f7b4242fc8e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713002bf55bf4d04a01ed94ed4f53c87"
I20260807 00:43:59.246840 26828 raft_consensus.cc:3037] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:59.246903 27225 raft_consensus.cc:3037] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:43:59.247634 26828 raft_consensus.cc:2454] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3524189c2254da0b804f7b4242fc8e1 in term 1.
I20260807 00:43:59.247859 27225 raft_consensus.cc:2454] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f3524189c2254da0b804f7b4242fc8e1 in term 1.
I20260807 00:43:59.248049 26918 leader_election.cc:302] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 492738822aa643c491d5f7a5c205de65, f3524189c2254da0b804f7b4242fc8e1; no voters:
I20260807 00:43:59.248210 27333 raft_consensus.cc:2781] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:43:59.248396 27333 raft_consensus.cc:686] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 1 LEADER]: Becoming Leader. State: Replica: f3524189c2254da0b804f7b4242fc8e1, State: Running, Role: LEADER
I20260807 00:43:59.248521 27333 consensus_queue.cc:237] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } }
I20260807 00:43:59.249575 26617 catalog_manager.cc:5495] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to f3524189c2254da0b804f7b4242fc8e1 (127.1.116.130). New cstate: current_term: 1 leader_uuid: "f3524189c2254da0b804f7b4242fc8e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3524189c2254da0b804f7b4242fc8e1" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 43665 } health_report { overall_health: HEALTHY } } }
I20260807 00:43:59.259675 27318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.257036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60112 (local address 127.1.116.130:43665)
0807 00:43:59.257120 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0807 00:43:59.257122 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:43:59.257133 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:43:59.257202 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:43:59.257203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:43:59.257227 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:43:59.257319 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:43:59.257469 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.257972 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.258849 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:43:59.259060 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:43:59.259138 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:43:59.259295 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:43:59.259418 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:43:59.259511 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:43:59.259537 (+ 26us) server_negotiation.cc:299] Negotiation successful
0807 00:43:59.259557 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:43:59.261739 26828 raft_consensus.cc:1261] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 1 FOLLOWER]: Refusing update from remote peer f3524189c2254da0b804f7b4242fc8e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:43:59.261757 27225 raft_consensus.cc:1261] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 1 FOLLOWER]: Refusing update from remote peer f3524189c2254da0b804f7b4242fc8e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:43:59.262133 27333 consensus_queue.cc:1035] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "713002bf55bf4d04a01ed94ed4f53c87" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 44501 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:43:59.262300 27338 consensus_queue.cc:1035] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "492738822aa643c491d5f7a5c205de65" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 35281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:43:59.265049 27347 mvcc.cc:205] Tried to move back new op lower bound from 7315715847212797952 to 7315715847162863616. Current Snapshot: MvccSnapshot[applied={T|T < 7315715847212797952}]
I20260807 00:43:59.267381 26617 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:43:59.267488 26617 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50844:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:43:59.268365 26617 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcdeccdb8f484beb9e5fa7fcd7bddab0: Sending DeleteTablet for 3 replicas of tablet 00d5e5dcc33346c5b955d03ef2df32d5
I20260807 00:43:59.268747 26994 tablet_service.cc:1514] Processing DeleteTablet for tablet 00d5e5dcc33346c5b955d03ef2df32d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:59 UTC) from {username='slave'} at 127.0.0.1:60106
I20260807 00:43:59.268729 27195 tablet_service.cc:1514] Processing DeleteTablet for tablet 00d5e5dcc33346c5b955d03ef2df32d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:59 UTC) from {username='slave'} at 127.0.0.1:57624
I20260807 00:43:59.268770 26801 tablet_service.cc:1514] Processing DeleteTablet for tablet 00d5e5dcc33346c5b955d03ef2df32d5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:43:59 UTC) from {username='slave'} at 127.0.0.1:41568
I20260807 00:43:59.269022 27358 tablet_replica.cc:330] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: stopping tablet replica
I20260807 00:43:59.269138 27362 tablet_replica.cc:330] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65: stopping tablet replica
I20260807 00:43:59.269142 27358 raft_consensus.cc:2229] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:43:59.269217 27362 raft_consensus.cc:2229] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:43:59.269244 27358 raft_consensus.cc:2258] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:59.269295 27362 raft_consensus.cc:2258] T 00d5e5dcc33346c5b955d03ef2df32d5 P 492738822aa643c491d5f7a5c205de65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:59.269412 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 26699
I20260807 00:43:59.269428 27360 tablet_replica.cc:330] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: stopping tablet replica
I20260807 00:43:59.286979 27360 raft_consensus.cc:2229] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:43:59.287201 27360 raft_consensus.cc:2258] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:43:59.287916 27358 ts_tablet_manager.cc:1905] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:43:59.287973 27360 ts_tablet_manager.cc:1905] T 00d5e5dcc33346c5b955d03ef2df32d5 P f3524189c2254da0b804f7b4242fc8e1: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 00:43:59.288254 26597 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:35281 (error 108)
I20260807 00:43:59.288873 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 26892
I20260807 00:43:59.292721 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.292557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:35281 (local address 127.0.0.1:41576)
0807 00:43:59.292620 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.292678 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:35281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 00:43:59.292959 26597 catalog_manager.cc:4584] TS 492738822aa643c491d5f7a5c205de65 (127.1.116.129:35281): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 00d5e5dcc33346c5b955d03ef2df32d5: Network error: Client connection negotiation failed: client connection to 127.1.116.129:35281: connect: Connection refused (error 111)
I20260807 00:43:59.293337 27358 ts_tablet_manager.cc:1918] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:43:59.293409 27358 log.cc:1196] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/00d5e5dcc33346c5b955d03ef2df32d5
I20260807 00:43:59.293665 27358 ts_tablet_manager.cc:1939] T 00d5e5dcc33346c5b955d03ef2df32d5 P 713002bf55bf4d04a01ed94ed4f53c87: Deleting consensus metadata
I20260807 00:43:59.294198 26597 catalog_manager.cc:4841] TS 713002bf55bf4d04a01ed94ed4f53c87 (127.1.116.131:44501): tablet 00d5e5dcc33346c5b955d03ef2df32d5 (table test-table [id=9c071eefd9294332bfaef9f2d144225c]) successfully deleted
I20260807 00:43:59.297866 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:43:59.297738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:43665 (local address 127.0.0.1:60116)
0807 00:43:59.297808 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 00:43:59.297841 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.130:43665: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 00:43:59.297890 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 27103
I20260807 00:43:59.308817 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 26571
2026-08-07T00:43:59Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (774 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:43:59 dist-test-slave-h6n0 krb5kdc[27367](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:43:59 dist-test-slave-h6n0 krb5kdc[27367](info): set up 2 sockets
Aug 07 00:43:59 dist-test-slave-h6n0 krb5kdc[27367](info): commencing operation
krb5kdc: starting...
W20260807 00:44:00.577934 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:00 dist-test-slave-h6n0 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:00.711190 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43839
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:45267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:43839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:00.803400 27387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:00.803623 27387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:00.803646 27387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:00.804848 27387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:00.804894 27387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:00.804908 27387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:00.804921 27387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:00.804934 27387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:00.806473 27387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:43839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:00.806780 27387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:00.807015 27387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:00.812407 27402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:00.812404 27394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:00.812404 27396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:00.812801 27387 server_base.cc:1034] running on GCE node
I20260807 00:44:00.813076 27387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:00.813397 27387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:00.814615 27387 hybrid_clock.cc:648] HybridClock initialized: now 1786063440814568 us; error 52 us; skew 500 ppm
Aug 07 00:44:00 dist-test-slave-h6n0 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063440, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:00.817441 27387 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:00.817970 27387 webserver.cc:460] Webserver started at http://127.1.116.190:39699/ using document root <none> and password file <none>
I20260807 00:44:00.818223 27387 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:00.818272 27387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:00.818466 27387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:00.819653 27387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "6f20516194c64dee841957d2a52c1195"
format_stamp: "Formatted at 2026-08-07 00:44:00 on dist-test-slave-h6n0"
server_key: "5305e27eb268065baaac30a366282f49"
server_key_iv: "065f8f54199e2cfed9c993b4c7bd1a10"
server_key_version: "encryptionkey@0"
I20260807 00:44:00.820057 27387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "6f20516194c64dee841957d2a52c1195"
format_stamp: "Formatted at 2026-08-07 00:44:00 on dist-test-slave-h6n0"
server_key: "5305e27eb268065baaac30a366282f49"
server_key_iv: "065f8f54199e2cfed9c993b4c7bd1a10"
server_key_version: "encryptionkey@0"
I20260807 00:44:00.821671 27387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:00.822569 27416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:00.822804 27387 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:00.822894 27387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "6f20516194c64dee841957d2a52c1195"
format_stamp: "Formatted at 2026-08-07 00:44:00 on dist-test-slave-h6n0"
server_key: "5305e27eb268065baaac30a366282f49"
server_key_iv: "065f8f54199e2cfed9c993b4c7bd1a10"
server_key_version: "encryptionkey@0"
I20260807 00:44:00.822999 27387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:00.838002 27387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:00.840169 27387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:00.840341 27387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:00.845364 27387 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:43839
I20260807 00:44:00.845427 27501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:43839 every 8 connection(s)
I20260807 00:44:00.845818 27387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:00.846860 27502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:00.847474 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 27387
I20260807 00:44:00.847582 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:00.847854 1490 external_mini_cluster.cc:1428] Setting key 792fc85498422c7180861a894c020563
I20260807 00:44:00.850121 27502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195: Bootstrap starting.
I20260807 00:44:00.850894 27502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:00.851230 27502 log.cc:824] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:00.851969 27502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195: No bootstrap required, opened a new log
I20260807 00:44:00.854043 27502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f20516194c64dee841957d2a52c1195" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43839 } }
I20260807 00:44:00.854250 27502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:00.854300 27502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f20516194c64dee841957d2a52c1195, State: Initialized, Role: FOLLOWER
I20260807 00:44:00.854442 27502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f20516194c64dee841957d2a52c1195" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43839 } }
I20260807 00:44:00.854568 27502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:00.854616 27502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:00.854672 27502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 0 FOLLOWER]: Advancing to term 1
Aug 07 00:44:00 dist-test-slave-h6n0 krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:00.855348 27502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f20516194c64dee841957d2a52c1195" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43839 } }
I20260807 00:44:00.855448 27502 leader_election.cc:302] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f20516194c64dee841957d2a52c1195; no voters:
I20260807 00:44:00.855667 27502 leader_election.cc:288] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:00.855695 27511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:00.855844 27511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [term 1 LEADER]: Becoming Leader. State: Replica: 6f20516194c64dee841957d2a52c1195, State: Running, Role: LEADER
I20260807 00:44:00.855948 27511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f20516194c64dee841957d2a52c1195" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43839 } }
I20260807 00:44:00.856001 27502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:00.856319 27511 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f20516194c64dee841957d2a52c1195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f20516194c64dee841957d2a52c1195" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43839 } } }
I20260807 00:44:00.856401 27511 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:00.856408 27513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f20516194c64dee841957d2a52c1195. Latest consensus state: current_term: 1 leader_uuid: "6f20516194c64dee841957d2a52c1195" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f20516194c64dee841957d2a52c1195" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43839 } } }
I20260807 00:44:00.856568 27513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:00.856832 27518 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:00.858336 27518 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:00.860261 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:00.848423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.1.116.190:43839)
0807 00:44:00.848668 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:00.848678 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:00.848707 (+ 29us) server_negotiation.cc:407] Connection header received
0807 00:44:00.849312 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:00.849319 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:00.849371 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:00.849533 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:00.850248 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:00.851348 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:00.852149 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:00.852583 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:00.852709 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:00.855297 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:00.855300 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:00.855301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:00.855313 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:00.858119 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:00.858598 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:00.858601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:00.858601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:00.858706 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:00.859048 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:00.859050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:00.859050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:00.859378 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:00.859567 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:00.859898 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 00:44:00.860061 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
I20260807 00:44:00.860387 27518 catalog_manager.cc:1347] Generated new cluster ID: b0075a58dfee458193067aec8ad4f64d
I20260807 00:44:00.860435 27518 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:00.871151 27518 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:00.871690 27518 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:00.879339 27518 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195: Generated new TSK 0
I20260807 00:44:00.879525 27518 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:44:00.914520 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43839
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:45267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:01.007815 27539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:01.008105 27539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:01.008152 27539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:01.009361 27539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:01.009428 27539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:01.009473 27539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:44:01.011052 27539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:01.011349 27539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:01.011607 27539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:01.017244 27548 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:01.017227 27550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:01.017381 27539 server_base.cc:1034] running on GCE node
W20260807 00:44:01.017203 27555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:01.017753 27539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:01.018033 27539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:01.019191 27539 hybrid_clock.cc:648] HybridClock initialized: now 1786063441019190 us; error 43 us; skew 500 ppm
Aug 07 00:44:01 dist-test-slave-h6n0 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:01.022367 27539 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:44:01.022981 27539 webserver.cc:460] Webserver started at http://127.1.116.129:38049/ using document root <none> and password file <none>
I20260807 00:44:01.023248 27539 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:01.023419 27539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:01.023613 27539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:01.025203 27539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "874ffdbdf8334aac93baf5cdbf215b88"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "b3540bed86e73147e448559d778e4d1b"
server_key_iv: "16c78cbfbc85abb84fa0b8dea0d8cc54"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.025671 27539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "874ffdbdf8334aac93baf5cdbf215b88"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "b3540bed86e73147e448559d778e4d1b"
server_key_iv: "16c78cbfbc85abb84fa0b8dea0d8cc54"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.027377 27539 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260807 00:44:01.028316 27568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.028544 27539 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:01.028647 27539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "874ffdbdf8334aac93baf5cdbf215b88"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "b3540bed86e73147e448559d778e4d1b"
server_key_iv: "16c78cbfbc85abb84fa0b8dea0d8cc54"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.028715 27539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:01.051397 27539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:01.053803 27539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:01.053931 27539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:01.054185 27539 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:01.054601 27539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:01.054644 27539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.054675 27539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:01.054689 27539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.062753 27539 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:45831
I20260807 00:44:01.062865 27749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:45831 every 8 connection(s)
I20260807 00:44:01.063218 27539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
Aug 07 00:44:01 dist-test-slave-h6n0 krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:01.071115 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 27539
I20260807 00:44:01.071296 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:44:01.071548 1490 external_mini_cluster.cc:1428] Setting key 997e21c7accd1b6dce627fb75da46731
I20260807 00:44:01.074581 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.064313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:59775 (local address 127.1.116.190:43839)
0807 00:44:01.064385 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.064388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.064486 (+ 98us) server_negotiation.cc:407] Connection header received
0807 00:44:01.065160 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.065161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.065174 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.065242 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:01.066279 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.066970 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.068082 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.068331 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.068403 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.070838 (+ 2435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:01.070840 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:01.070840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:01.070848 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:01.072783 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.073355 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.073356 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.073357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.073427 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.073721 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.073722 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.073723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.073863 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:01.073952 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.074292 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.074428 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 00:44:01.074867 27754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.064030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43839 (local address 127.1.116.129:59775)
0807 00:44:01.064359 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.064403 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.064994 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.065319 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.065323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.065952 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:01.066165 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.066170 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.067097 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.067098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.067948 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.067950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.068426 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.068427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.068533 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.069117 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:01.069126 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:01.070722 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:01.072902 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:01.072913 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:01.072916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:01.073262 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:01.073512 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:01.073513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:01.073513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:01.073647 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:01.073941 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:01.073949 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:01.074134 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.074265 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.074474 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":81,"threads_started":1}
I20260807 00:44:01.075519 27751 heartbeater.cc:347] Connected to a master server at 127.1.116.190:43839
I20260807 00:44:01.075690 27751 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:01.075932 27751 heartbeater.cc:511] Master 127.1.116.190:43839 requested a full tablet report, sending...
I20260807 00:44:01.076607 27443 ts_manager.cc:194] Registered new tserver with Master: 874ffdbdf8334aac93baf5cdbf215b88 (127.1.116.129:45831)
I20260807 00:44:01.077333 27443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:59775
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:44:01.097579 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43839
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:45267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:01.188230 27761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:01.188416 27761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:01.188439 27761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:01.189576 27761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:01.189623 27761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:01.189649 27761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:44:01.191143 27761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:01.191383 27761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:01.191635 27761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:44:01.196916 27761 server_base.cc:1034] running on GCE node
W20260807 00:44:01.196868 27767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:01.196931 27766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:01.196878 27772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:01.197386 27761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:01.197592 27761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:01.198773 27761 hybrid_clock.cc:648] HybridClock initialized: now 1786063441198756 us; error 35 us; skew 500 ppm
Aug 07 00:44:01 dist-test-slave-h6n0 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:01.201390 27761 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:44:01.201869 27761 webserver.cc:460] Webserver started at http://127.1.116.130:35069/ using document root <none> and password file <none>
I20260807 00:44:01.202112 27761 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:01.202181 27761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:01.202337 27761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:01.203485 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "413108eb5da64b5cae22342765dbb122"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "02a443baece2a7c7e94fd179ba2db160"
server_key_iv: "0c3a45c6efb2408f9c72320151be1828"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.203848 27761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "413108eb5da64b5cae22342765dbb122"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "02a443baece2a7c7e94fd179ba2db160"
server_key_iv: "0c3a45c6efb2408f9c72320151be1828"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.205253 27761 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:44:01.206073 27786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.206344 27761 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:01.206451 27761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "413108eb5da64b5cae22342765dbb122"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "02a443baece2a7c7e94fd179ba2db160"
server_key_iv: "0c3a45c6efb2408f9c72320151be1828"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.206570 27761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:01.227519 27761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:01.229730 27761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:01.230142 27761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:01.230599 27761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:01.231189 27761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:01.231292 27761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.231367 27761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:01.231398 27761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.239607 27761 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:37195
I20260807 00:44:01.239658 27960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:37195 every 8 connection(s)
I20260807 00:44:01.240036 27761 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:44:01.244452 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 27761
I20260807 00:44:01.244627 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:44:01.244903 1490 external_mini_cluster.cc:1428] Setting key 288e6990c6c88dedc365fb5390079b4a
Aug 07 00:44:01 dist-test-slave-h6n0 krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:01.251533 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.240735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43839 (local address 127.1.116.130:56181)
0807 00:44:01.240916 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.240956 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.241445 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.241707 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.241710 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.242267 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:01.242453 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.242458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.243307 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.243308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.244133 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.244136 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.244717 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.244719 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.245074 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.245925 (+ 851us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:01.245937 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:01.247896 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:01.249896 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:01.249903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:01.249905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:01.250162 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:01.250435 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:01.250436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:01.250437 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:01.250629 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:01.250987 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:01.250994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:01.251118 (+ 124us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.251223 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.251359 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":104,"threads_started":1}
I20260807 00:44:01.251538 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.240770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:56181 (local address 127.1.116.190:43839)
0807 00:44:01.240850 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.240852 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.241068 (+ 216us) server_negotiation.cc:407] Connection header received
0807 00:44:01.241546 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.241548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.241560 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.241655 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:01.242600 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.243209 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.244252 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.244589 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.244686 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.247997 (+ 3311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:01.247999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:01.248000 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:01.248009 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:01.249787 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.250256 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.250258 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.250259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.250331 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.250712 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.250713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.250713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.250884 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:01.250966 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.251270 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.251379 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:44:01.252173 27961 heartbeater.cc:347] Connected to a master server at 127.1.116.190:43839
I20260807 00:44:01.252274 27961 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:01.252461 27961 heartbeater.cc:511] Master 127.1.116.190:43839 requested a full tablet report, sending...
I20260807 00:44:01.252902 27443 ts_manager.cc:194] Registered new tserver with Master: 413108eb5da64b5cae22342765dbb122 (127.1.116.130:37195)
I20260807 00:44:01.253294 27443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:56181
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:44:01.274690 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43839
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:45267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:01.366725 27968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:01.366982 27968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:01.367018 27968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:01.368202 27968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:01.368270 27968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:01.368317 27968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:44:01.369899 27968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:45267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:01.370191 27968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:01.370443 27968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:01.376345 27980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:01.376327 27974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:01.376356 27973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:01.376628 27968 server_base.cc:1034] running on GCE node
I20260807 00:44:01.376852 27968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:01.377081 27968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:01.378237 27968 hybrid_clock.cc:648] HybridClock initialized: now 1786063441378216 us; error 53 us; skew 500 ppm
Aug 07 00:44:01 dist-test-slave-h6n0 krb5kdc[27367](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:01.384274 27968 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:44:01.384747 27968 webserver.cc:460] Webserver started at http://127.1.116.131:41075/ using document root <none> and password file <none>
I20260807 00:44:01.384999 27968 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:01.385069 27968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:01.385226 27968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:01.386317 27968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "69e701c01775408b9c1f3f7e165f9758"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "c740c58402926297b8c193e7e0ec9540"
server_key_iv: "e0dbeacf31a56919d7dc33e54b041240"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.386718 27968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "69e701c01775408b9c1f3f7e165f9758"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "c740c58402926297b8c193e7e0ec9540"
server_key_iv: "e0dbeacf31a56919d7dc33e54b041240"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.388124 27968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:01.388936 27991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.389158 27968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:01.389276 27968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "69e701c01775408b9c1f3f7e165f9758"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "c740c58402926297b8c193e7e0ec9540"
server_key_iv: "e0dbeacf31a56919d7dc33e54b041240"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.389362 27968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:01.399617 27968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:01.401494 27968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:01.401646 27968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:01.401856 27968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:01.402215 27968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:01.402278 27968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.402319 27968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:01.402347 27968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.409221 27968 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:41009
I20260807 00:44:01.409308 28181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:41009 every 8 connection(s)
I20260807 00:44:01.409611 27968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:44:01.410993 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 27968
I20260807 00:44:01.411104 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:44:01.411293 1490 external_mini_cluster.cc:1428] Setting key ed6aefae28b848bd92ebb9cdcac6bf6a
Aug 07 00:44:01 dist-test-slave-h6n0 krb5kdc[27367](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063441, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:01.419584 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.410495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:34473 (local address 127.1.116.190:43839)
0807 00:44:01.410624 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.410629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.410905 (+ 276us) server_negotiation.cc:407] Connection header received
0807 00:44:01.411608 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.411610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.411669 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.411748 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:01.412832 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.413382 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.414405 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.414689 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.414747 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.416949 (+ 2202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:01.416951 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:01.416951 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:01.416959 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:01.418332 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.418791 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.418792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.418792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.418847 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.419061 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.419061 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.419062 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.419160 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:01.419210 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.419439 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.419507 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 00:44:01.419672 28186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.410485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:43839 (local address 127.1.116.131:34473)
0807 00:44:01.410742 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.410779 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.411469 (+ 690us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.411834 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.411838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.412559 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:01.412734 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.412739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.413520 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.413521 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.414321 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.414323 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.414768 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.414769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.414870 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.415321 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:01.415328 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:01.416782 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:01.418415 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:01.418421 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:01.418422 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:01.418708 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:01.418917 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:01.418917 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:01.418918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:01.419013 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:01.419217 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:01.419221 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:01.419319 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.419407 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.419519 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":171,"threads_started":1}
I20260807 00:44:01.420347 28182 heartbeater.cc:347] Connected to a master server at 127.1.116.190:43839
I20260807 00:44:01.420433 28182 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:01.420565 28182 heartbeater.cc:511] Master 127.1.116.190:43839 requested a full tablet report, sending...
I20260807 00:44:01.420958 27443 ts_manager.cc:194] Registered new tserver with Master: 69e701c01775408b9c1f3f7e165f9758 (127.1.116.131:41009)
I20260807 00:44:01.421380 27443 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:34473
I20260807 00:44:01.423482 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:44:01.431836 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.425090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57512 (local address 127.1.116.190:43839)
0807 00:44:01.425151 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.425153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.425303 (+ 150us) server_negotiation.cc:407] Connection header received
0807 00:44:01.425376 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.425377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.425388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.425439 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:01.426131 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.426743 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.427487 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.427693 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.427770 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.429130 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:01.429132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:01.429132 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:01.429139 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:01.430405 (+ 1266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.430869 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.430871 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.430871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.430941 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:01.431222 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:01.431223 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:01.431223 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:01.431365 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:01.431501 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.431649 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.431717 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 00:44:01.433557 27443 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57512:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:44:01.435348 27443 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:44:01.443145 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.438458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50592 (local address 127.1.116.130:37195)
0807 00:44:01.438717 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.438723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.438835 (+ 112us) server_negotiation.cc:407] Connection header received
0807 00:44:01.438938 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.438941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.438963 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.439073 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:01.439400 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.440601 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.442002 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.442677 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.442760 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.442813 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.443007 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.443057 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":116,"threads_started":1}
I20260807 00:44:01.443166 28197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.438226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:37195 (local address 127.0.0.1:50592)
0807 00:44:01.438427 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.438490 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.438590 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.439080 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.439084 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.439087 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:01.439274 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.439281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.440721 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.440722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.441870 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.441873 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.442776 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.442777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.442867 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.442883 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.442971 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.443044 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":35,"thread_start_us":103,"threads_started":1}
I20260807 00:44:01.443488 28201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.438226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:45831 (local address 127.0.0.1:38956)
0807 00:44:01.439285 (+ 1059us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.439300 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.439379 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.439817 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.439818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.439820 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:01.439915 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.439917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.440860 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.440861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.442104 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.442107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.443164 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.443165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.443230 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.443234 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.443373 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.443416 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961,"mutex_wait_us":6,"thread_start_us":67,"threads_started":1}
I20260807 00:44:01.443614 28199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.438246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:41009 (local address 127.0.0.1:53142)
0807 00:44:01.439498 (+ 1252us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.439505 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.439549 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.439826 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.439827 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.439829 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:01.439916 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.439918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.441240 (+ 1322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.441241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.442296 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.442298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.443336 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.443338 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.443405 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.443411 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.443461 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.443523 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1224,"mutex_wait_us":29,"thread_start_us":120,"threads_started":1}
I20260807 00:44:01.443876 28205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.438569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.1.116.131:41009)
0807 00:44:01.438799 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.438804 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.439576 (+ 772us) server_negotiation.cc:407] Connection header received
0807 00:44:01.439627 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.439630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.439654 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.439745 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:01.439994 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.441002 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.442400 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.443144 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.443224 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.443290 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.443722 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.443778 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
I20260807 00:44:01.443872 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.438517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.1.116.129:45831)
0807 00:44:01.438737 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.438740 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.439609 (+ 869us) server_negotiation.cc:407] Connection header received
0807 00:44:01.439650 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.439652 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.439665 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.439807 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:01.440000 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.440741 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.442228 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.443055 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.443598 (+ 543us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.443689 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.443719 (+ 30us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.443768 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":113,"threads_started":1}
I20260807 00:44:01.444509 28066 tablet_service.cc:1467] Processing CreateTablet for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c (DEFAULT_TABLE table=test-table [id=4f39e3857d754e9a8fa50119bfba75a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:01.444682 27642 tablet_service.cc:1467] Processing CreateTablet for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c (DEFAULT_TABLE table=test-table [id=4f39e3857d754e9a8fa50119bfba75a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:01.444669 27859 tablet_service.cc:1467] Processing CreateTablet for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c (DEFAULT_TABLE table=test-table [id=4f39e3857d754e9a8fa50119bfba75a9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:01.444849 28066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:01.445084 27859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:01.445084 27642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:01.447537 28208 tablet_bootstrap.cc:492] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Bootstrap starting.
I20260807 00:44:01.448205 28208 tablet_bootstrap.cc:654] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:01.448697 28208 log.cc:824] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:01.448966 28214 tablet_bootstrap.cc:492] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: Bootstrap starting.
I20260807 00:44:01.449730 28208 tablet_bootstrap.cc:492] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: No bootstrap required, opened a new log
I20260807 00:44:01.449731 28214 tablet_bootstrap.cc:654] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:01.449918 28208 ts_tablet_manager.cc:1397] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:44:01.450448 28214 log.cc:824] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:01.450717 28213 tablet_bootstrap.cc:492] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Bootstrap starting.
I20260807 00:44:01.451452 28213 tablet_bootstrap.cc:654] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:01.451498 28214 tablet_bootstrap.cc:492] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: No bootstrap required, opened a new log
I20260807 00:44:01.451582 28214 ts_tablet_manager.cc:1397] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 00:44:01.452023 28213 log.cc:824] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:01.452224 28208 raft_consensus.cc:348] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.452414 28208 raft_consensus.cc:374] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:01.452455 28208 raft_consensus.cc:729] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e701c01775408b9c1f3f7e165f9758, State: Initialized, Role: FOLLOWER
I20260807 00:44:01.452605 28208 consensus_queue.cc:260] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.452847 28213 tablet_bootstrap.cc:492] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: No bootstrap required, opened a new log
I20260807 00:44:01.452880 28208 ts_tablet_manager.cc:1428] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:01.452922 28213 ts_tablet_manager.cc:1397] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:44:01.453004 28182 heartbeater.cc:503] Master 127.1.116.190:43839 was elected leader, sending a full tablet report...
I20260807 00:44:01.453677 28214 raft_consensus.cc:348] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.453814 28214 raft_consensus.cc:374] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:01.453835 28214 raft_consensus.cc:729] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 874ffdbdf8334aac93baf5cdbf215b88, State: Initialized, Role: FOLLOWER
I20260807 00:44:01.453945 28214 consensus_queue.cc:260] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.454267 28214 ts_tablet_manager.cc:1428] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260807 00:44:01.454288 27751 heartbeater.cc:503] Master 127.1.116.190:43839 was elected leader, sending a full tablet report...
I20260807 00:44:01.455224 28213 raft_consensus.cc:348] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.455425 28213 raft_consensus.cc:374] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:01.455475 28213 raft_consensus.cc:729] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413108eb5da64b5cae22342765dbb122, State: Initialized, Role: FOLLOWER
I20260807 00:44:01.455605 28213 consensus_queue.cc:260] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.455914 28213 ts_tablet_manager.cc:1428] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:01.455996 27961 heartbeater.cc:503] Master 127.1.116.190:43839 was elected leader, sending a full tablet report...
W20260807 00:44:01.490761 27962 tablet.cc:2369] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:44:01.564038 27752 tablet.cc:2369] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:44:01.581736 28223 raft_consensus.cc:482] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:44:01.581923 28223 raft_consensus.cc:504] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.582515 28223 leader_election.cc:288] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 874ffdbdf8334aac93baf5cdbf215b88 (127.1.116.129:45831), 69e701c01775408b9c1f3f7e165f9758 (127.1.116.131:41009)
I20260807 00:44:01.587034 28205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.582952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:32923 (local address 127.1.116.131:41009)
0807 00:44:01.583029 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.583031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.583247 (+ 216us) server_negotiation.cc:407] Connection header received
0807 00:44:01.583282 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.583283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.583317 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.583381 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:01.583530 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.584322 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.585848 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.586554 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.586633 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.586647 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.586896 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.586952 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260807 00:44:01.587034 28203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.582859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:59625 (local address 127.1.116.129:45831)
0807 00:44:01.582958 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.582961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.582978 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:44:01.583021 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.583022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.583035 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.583115 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:01.583338 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.584270 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.585843 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.586495 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.586677 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.586688 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.586896 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.586949 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:44:01.587025 27963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.582778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:45831 (local address 127.1.116.130:59625)
0807 00:44:01.582839 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.582853 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.582908 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.583148 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.583149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.583152 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:01.583238 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.583241 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.584388 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.584389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.585727 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.585729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.586693 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.586694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.586776 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.586779 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.586842 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.586896 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":53}
I20260807 00:44:01.587033 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.582844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:41009 (local address 127.1.116.130:32923)
0807 00:44:01.583121 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.583134 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:01.583201 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:01.583395 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:01.583396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:01.583397 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:01.583452 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.583453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.584407 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.584407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.585727 (+ 1320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:01.585729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.586653 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.586654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:01.586762 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.586767 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:44:01.586842 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:44:01.586896 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":44,"thread_start_us":163,"threads_started":1}
I20260807 00:44:01.587527 27682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e73ffb0e3b463b9e35b1e9dae7e73c" candidate_uuid: "413108eb5da64b5cae22342765dbb122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ffdbdf8334aac93baf5cdbf215b88" is_pre_election: true
I20260807 00:44:01.587671 28102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e73ffb0e3b463b9e35b1e9dae7e73c" candidate_uuid: "413108eb5da64b5cae22342765dbb122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e701c01775408b9c1f3f7e165f9758" is_pre_election: true
I20260807 00:44:01.587733 27682 raft_consensus.cc:2454] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413108eb5da64b5cae22342765dbb122 in term 0.
I20260807 00:44:01.587819 28102 raft_consensus.cc:2454] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 413108eb5da64b5cae22342765dbb122 in term 0.
I20260807 00:44:01.588037 27791 leader_election.cc:302] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413108eb5da64b5cae22342765dbb122, 874ffdbdf8334aac93baf5cdbf215b88; no voters:
I20260807 00:44:01.588340 28223 raft_consensus.cc:2781] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:44:01.588423 28223 raft_consensus.cc:482] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:44:01.588454 28223 raft_consensus.cc:3037] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:01.589246 28223 raft_consensus.cc:504] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.589448 28223 leader_election.cc:288] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [CANDIDATE]: Term 1 election: Requested vote from peers 874ffdbdf8334aac93baf5cdbf215b88 (127.1.116.129:45831), 69e701c01775408b9c1f3f7e165f9758 (127.1.116.131:41009)
I20260807 00:44:01.589628 27682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e73ffb0e3b463b9e35b1e9dae7e73c" candidate_uuid: "413108eb5da64b5cae22342765dbb122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "874ffdbdf8334aac93baf5cdbf215b88"
I20260807 00:44:01.589638 28102 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a8e73ffb0e3b463b9e35b1e9dae7e73c" candidate_uuid: "413108eb5da64b5cae22342765dbb122" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e701c01775408b9c1f3f7e165f9758"
I20260807 00:44:01.589756 28102 raft_consensus.cc:3037] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:01.589747 27682 raft_consensus.cc:3037] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:01.590898 28102 raft_consensus.cc:2454] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413108eb5da64b5cae22342765dbb122 in term 1.
I20260807 00:44:01.590898 27682 raft_consensus.cc:2454] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 413108eb5da64b5cae22342765dbb122 in term 1.
I20260807 00:44:01.591149 27789 leader_election.cc:302] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 413108eb5da64b5cae22342765dbb122, 69e701c01775408b9c1f3f7e165f9758; no voters:
I20260807 00:44:01.591257 28223 raft_consensus.cc:2781] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:01.591418 28223 raft_consensus.cc:686] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 1 LEADER]: Becoming Leader. State: Replica: 413108eb5da64b5cae22342765dbb122, State: Running, Role: LEADER
I20260807 00:44:01.591542 28223 consensus_queue.cc:237] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } }
I20260807 00:44:01.592495 27443 catalog_manager.cc:5495] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 reported cstate change: term changed from 0 to 1, leader changed from <none> to 413108eb5da64b5cae22342765dbb122 (127.1.116.130). New cstate: current_term: 1 leader_uuid: "413108eb5da64b5cae22342765dbb122" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "413108eb5da64b5cae22342765dbb122" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 37195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 } health_report { overall_health: UNKNOWN } } }
I20260807 00:44:01.628317 28202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.625521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.1.116.130:37195)
0807 00:44:01.625576 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.625579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.625590 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:44:01.625671 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.625671 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.625708 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.625797 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:01.626103 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.626615 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.627522 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.627718 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.627797 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.627937 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:01.628043 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:01.628166 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.628225 (+ 59us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.628236 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 00:44:01.630681 28102 raft_consensus.cc:1261] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 1 FOLLOWER]: Refusing update from remote peer 413108eb5da64b5cae22342765dbb122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:01.630746 27682 raft_consensus.cc:1261] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 1 FOLLOWER]: Refusing update from remote peer 413108eb5da64b5cae22342765dbb122: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:01.631129 28223 consensus_queue.cc:1035] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69e701c01775408b9c1f3f7e165f9758" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:01.631357 28225 consensus_queue.cc:1035] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [LEADER]: Connected to new peer: Peer: permanent_uuid: "874ffdbdf8334aac93baf5cdbf215b88" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 45831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:01.634200 28228 mvcc.cc:205] Tried to move back new op lower bound from 7315715856914845696 to 7315715856759619584. Current Snapshot: MvccSnapshot[applied={T|T < 7315715856914845696}]
I20260807 00:44:01.635017 28229 mvcc.cc:205] Tried to move back new op lower bound from 7315715856914845696 to 7315715856759619584. Current Snapshot: MvccSnapshot[applied={T|T < 7315715856914845696}]
I20260807 00:44:01.637439 27443 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:01.637547 27443 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57512:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:01.638629 27443 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6f20516194c64dee841957d2a52c1195: Sending DeleteTablet for 3 replicas of tablet a8e73ffb0e3b463b9e35b1e9dae7e73c
I20260807 00:44:01.639004 27859 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:01 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:50592
I20260807 00:44:01.638998 27642 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:01 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:38956
I20260807 00:44:01.639070 28066 tablet_service.cc:1514] Processing DeleteTablet for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:01 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:53142
I20260807 00:44:01.639451 28251 tablet_replica.cc:330] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88: stopping tablet replica
I20260807 00:44:01.639582 28251 raft_consensus.cc:2229] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 874ffdbdf8334aac93baf5cdbf215b88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:44:01.639604 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 27539
I20260807 00:44:01.647336 28244 tablet_replica.cc:330] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: stopping tablet replica
I20260807 00:44:01.647475 28244 raft_consensus.cc:2229] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:44:01.647594 28244 raft_consensus.cc:2258] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:01.651978 28244 ts_tablet_manager.cc:1905] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:44:01.654001 28244 ts_tablet_manager.cc:1918] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:44:01.654072 28244 log.cc:1196] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/a8e73ffb0e3b463b9e35b1e9dae7e73c
I20260807 00:44:01.654383 28244 ts_tablet_manager.cc:1939] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 413108eb5da64b5cae22342765dbb122: Deleting consensus metadata
I20260807 00:44:01.654979 28246 tablet_replica.cc:330] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: stopping tablet replica
I20260807 00:44:01.655102 28246 raft_consensus.cc:2229] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:44:01.655069 27417 catalog_manager.cc:4841] TS 413108eb5da64b5cae22342765dbb122 (127.1.116.130:37195): tablet a8e73ffb0e3b463b9e35b1e9dae7e73c (table test-table [id=4f39e3857d754e9a8fa50119bfba75a9]) successfully deleted
I20260807 00:44:01.655222 28246 raft_consensus.cc:2258] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:01.661585 28246 ts_tablet_manager.cc:1905] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:44:01.663595 28246 ts_tablet_manager.cc:1918] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:44:01.663676 28246 log.cc:1196] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/a8e73ffb0e3b463b9e35b1e9dae7e73c
I20260807 00:44:01.664031 28246 ts_tablet_manager.cc:1939] T a8e73ffb0e3b463b9e35b1e9dae7e73c P 69e701c01775408b9c1f3f7e165f9758: Deleting consensus metadata
I20260807 00:44:01.664620 27418 catalog_manager.cc:4841] TS 69e701c01775408b9c1f3f7e165f9758 (127.1.116.131:41009): tablet a8e73ffb0e3b463b9e35b1e9dae7e73c (table test-table [id=4f39e3857d754e9a8fa50119bfba75a9]) successfully deleted
W20260807 00:44:01.671522 27422 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:45831 (error 108)
I20260807 00:44:01.671950 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 27761
I20260807 00:44:01.676980 28199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.672142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:45831 (local address 127.0.0.1:38960)
0807 00:44:01.676854 (+ 4712us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:01.676921 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:45831: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4673}
W20260807 00:44:01.677284 27422 catalog_manager.cc:4584] TS 874ffdbdf8334aac93baf5cdbf215b88 (127.1.116.129:45831): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a8e73ffb0e3b463b9e35b1e9dae7e73c: Network error: Client connection negotiation failed: client connection to 127.1.116.129:45831: connect: Connection refused (error 111)
I20260807 00:44:01.683662 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 27968
I20260807 00:44:01.694021 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 27387
2026-08-07T00:44:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2382 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T00:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:01Z Disabled control of system clock
I20260807 00:44:01.733059 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41799
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:40693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:41799
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:44:01.824517 28260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:01.824800 28260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:01.824821 28260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:01.826094 28260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:01.826148 28260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:01.826161 28260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:01.826174 28260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:01.826186 28260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:01.827777 28260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:41799
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:41799
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:01.828101 28260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:01.828454 28260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:01.834286 28269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:01.834333 28267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:01.834463 28260 server_base.cc:1034] running on GCE node
W20260807 00:44:01.834273 28266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:01.834879 28260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:01.835259 28260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:01.836440 28260 hybrid_clock.cc:648] HybridClock initialized: now 1786063441836433 us; error 54 us; skew 500 ppm
I20260807 00:44:01.837807 28260 webserver.cc:460] Webserver started at http://127.1.116.190:45495/ using document root <none> and password file <none>
I20260807 00:44:01.838076 28260 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:01.838150 28260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:01.838310 28260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:01.839497 28260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "32b39005f5824650bc9280451065eeaa"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "e53219c6739a969aa47347fc675fc349"
server_key_iv: "d328923fbb4013e615ce94a8b28f63bf"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.839874 28260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "32b39005f5824650bc9280451065eeaa"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "e53219c6739a969aa47347fc675fc349"
server_key_iv: "d328923fbb4013e615ce94a8b28f63bf"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.841264 28260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:44:01.842118 28276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:01.842384 28260 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:01.842523 28260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "32b39005f5824650bc9280451065eeaa"
format_stamp: "Formatted at 2026-08-07 00:44:01 on dist-test-slave-h6n0"
server_key: "e53219c6739a969aa47347fc675fc349"
server_key_iv: "d328923fbb4013e615ce94a8b28f63bf"
server_key_version: "encryptionkey@0"
I20260807 00:44:01.842660 28260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:01.863063 28260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:01.863358 28260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:01.863525 28260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:01.868093 28260 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:41799
I20260807 00:44:01.868214 28357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:41799 every 8 connection(s)
I20260807 00:44:01.868530 28260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:01.869673 28358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:01.872530 28358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa: Bootstrap starting.
I20260807 00:44:01.873237 28358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:01.873587 28358 log.cc:824] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:01.874431 28358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa: No bootstrap required, opened a new log
I20260807 00:44:01.876273 28358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b39005f5824650bc9280451065eeaa" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41799 } }
I20260807 00:44:01.876434 28358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:01.876459 28358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32b39005f5824650bc9280451065eeaa, State: Initialized, Role: FOLLOWER
I20260807 00:44:01.876644 28358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b39005f5824650bc9280451065eeaa" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41799 } }
I20260807 00:44:01.876716 28358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:01.876740 28358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:01.876824 28358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:01.877408 28358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b39005f5824650bc9280451065eeaa" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41799 } }
I20260807 00:44:01.877506 28358 leader_election.cc:302] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32b39005f5824650bc9280451065eeaa; no voters:
I20260807 00:44:01.877735 28358 leader_election.cc:288] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:01.877842 28365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:01.878094 28358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:01.878100 28365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [term 1 LEADER]: Becoming Leader. State: Replica: 32b39005f5824650bc9280451065eeaa, State: Running, Role: LEADER
I20260807 00:44:01.878280 28365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b39005f5824650bc9280451065eeaa" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41799 } }
I20260807 00:44:01.878507 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 28260
I20260807 00:44:01.878647 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:01.878841 28366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32b39005f5824650bc9280451065eeaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b39005f5824650bc9280451065eeaa" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41799 } } }
I20260807 00:44:01.878930 28366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:01.878942 1490 external_mini_cluster.cc:1428] Setting key cf1833ec59b0bcb08e596dd64d75e963
I20260807 00:44:01.879379 28382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:01.879962 28369 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32b39005f5824650bc9280451065eeaa. Latest consensus state: current_term: 1 leader_uuid: "32b39005f5824650bc9280451065eeaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32b39005f5824650bc9280451065eeaa" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 41799 } } }
I20260807 00:44:01.880074 28369 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:01.880007 28382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:01.882155 28382 catalog_manager.cc:1347] Generated new cluster ID: 209302682aca43ab9f17c5d9124173e3
I20260807 00:44:01.882213 28382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:01.886102 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:01.879867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57096 (local address 127.1.116.190:41799)
0807 00:44:01.880252 (+ 385us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:01.880278 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:01.880323 (+ 45us) server_negotiation.cc:407] Connection header received
0807 00:44:01.880582 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:01.880592 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:01.880686 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:01.880833 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:01.881159 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.882572 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.883764 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:01.884401 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:01.884513 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:01.885357 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:01.885361 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:01.885362 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:01.885582 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:01.885626 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:01.885640 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:01.885711 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:01.885812 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 00:44:01.885889 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":84,"threads_started":1}
I20260807 00:44:01.889686 28382 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:01.890476 28382 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:01.900051 28382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa: Generated new TSK 0
I20260807 00:44:01.900285 28382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:01.907178 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41799
--builtin_ntp_servers=127.1.116.148:40693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:44:01.992862 28401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:01.993070 28401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:01.993104 28401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:01.994177 28401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:01.994240 28401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:01.994279 28401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:44:01.995652 28401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:01.995905 28401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:01.996196 28401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:02.001399 28409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:02.001398 28411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:02.001645 28413 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:02.002059 28401 server_base.cc:1034] running on GCE node
I20260807 00:44:02.002313 28401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:02.002625 28401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:02.003793 28401 hybrid_clock.cc:648] HybridClock initialized: now 1786063442003782 us; error 31 us; skew 500 ppm
I20260807 00:44:02.005136 28401 webserver.cc:460] Webserver started at http://127.1.116.129:45599/ using document root <none> and password file <none>
I20260807 00:44:02.005414 28401 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:02.005462 28401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:02.005602 28401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:02.006727 28401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "d0cbd652de91406985c0f090181be06b"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "5bcc1a5b6417d302b6942ecb5cb3d56b"
server_key_iv: "4bc84c22f5d0759fe824a64faf1f8e71"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.007066 28401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "d0cbd652de91406985c0f090181be06b"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "5bcc1a5b6417d302b6942ecb5cb3d56b"
server_key_iv: "4bc84c22f5d0759fe824a64faf1f8e71"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.008381 28401 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:44:02.009155 28427 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.009333 28401 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260807 00:44:02.009424 28401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "d0cbd652de91406985c0f090181be06b"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "5bcc1a5b6417d302b6942ecb5cb3d56b"
server_key_iv: "4bc84c22f5d0759fe824a64faf1f8e71"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.009514 28401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:02.034931 28401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:02.035203 28401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:02.035346 28401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:02.035589 28401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:02.035919 28401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:02.035975 28401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.036010 28401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:02.036036 28401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.042860 28401 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:37857
I20260807 00:44:02.043074 28609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:37857 every 8 connection(s)
I20260807 00:44:02.043251 28401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:44:02.044333 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 28401
I20260807 00:44:02.044456 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:44:02.044726 1490 external_mini_cluster.cc:1428] Setting key 71e630714e3df9289cbe04e17699ff41
I20260807 00:44:02.045859 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41799
--builtin_ntp_servers=127.1.116.148:40693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:44:02.049505 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.044116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:34775 (local address 127.1.116.190:41799)
0807 00:44:02.044231 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.044234 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.044401 (+ 167us) server_negotiation.cc:407] Connection header received
0807 00:44:02.044982 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.044984 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.045004 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.045060 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:02.045809 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.046584 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.047533 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.047836 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.047908 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.049134 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:02.049136 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:02.049136 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:02.049142 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:02.049165 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:02.049170 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:02.049243 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.049384 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.049433 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:44:02.049726 28614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.044025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41799 (local address 127.1.116.129:34775)
0807 00:44:02.044249 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.044287 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.044866 (+ 579us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.045148 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.045151 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.045161 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:02.045714 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.045719 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.046667 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.046668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.047430 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.047432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.047983 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.047984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.048157 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.048923 (+ 766us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:44:02.048924 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:02.048963 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:44:02.048992 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:02.049271 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:02.049278 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:02.049281 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.049372 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.049482 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
I20260807 00:44:02.050338 28610 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41799
I20260807 00:44:02.050491 28610 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:02.050762 28610 heartbeater.cc:511] Master 127.1.116.190:41799 requested a full tablet report, sending...
I20260807 00:44:02.051455 28300 ts_manager.cc:194] Registered new tserver with Master: d0cbd652de91406985c0f090181be06b (127.1.116.129:37857)
I20260807 00:44:02.052167 28300 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.129:34775
W20260807 00:44:02.132036 28615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:02.132424 28615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:02.132493 28615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:02.133742 28615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:02.133787 28615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:02.133813 28615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:44:02.135246 28615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:02.135493 28615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:02.135811 28615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:02.141186 28626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:02.141191 28624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:02.141162 28623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:02.141211 28615 server_base.cc:1034] running on GCE node
I20260807 00:44:02.141772 28615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:02.142057 28615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:02.143249 28615 hybrid_clock.cc:648] HybridClock initialized: now 1786063442143228 us; error 39 us; skew 500 ppm
I20260807 00:44:02.144575 28615 webserver.cc:460] Webserver started at http://127.1.116.130:45265/ using document root <none> and password file <none>
I20260807 00:44:02.144901 28615 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:02.144991 28615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:02.145152 28615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:02.146198 28615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "2f17da84a10b475a804f57d47e16a0a7"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "43b3daa93ddbb006a852a4a670187cc1"
server_key_iv: "fbed89517301610eff7e4d4c3586e261"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.146611 28615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "2f17da84a10b475a804f57d47e16a0a7"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "43b3daa93ddbb006a852a4a670187cc1"
server_key_iv: "fbed89517301610eff7e4d4c3586e261"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.148186 28615 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:02.149328 28636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.149665 28615 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:02.149796 28615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "2f17da84a10b475a804f57d47e16a0a7"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "43b3daa93ddbb006a852a4a670187cc1"
server_key_iv: "fbed89517301610eff7e4d4c3586e261"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.149883 28615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:02.159909 28615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:02.160198 28615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:02.160356 28615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:02.160588 28615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:02.160923 28615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:02.160980 28615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.161012 28615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:02.161039 28615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.168123 28615 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:42355
I20260807 00:44:02.168175 28819 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:42355 every 8 connection(s)
I20260807 00:44:02.168479 28615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:44:02.173591 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 28615
I20260807 00:44:02.173743 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:44:02.173995 1490 external_mini_cluster.cc:1428] Setting key 6999f08317f19a2c82788e8c5a3256eb
I20260807 00:44:02.174507 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.169392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:40985 (local address 127.1.116.190:41799)
0807 00:44:02.169482 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.169485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.169842 (+ 357us) server_negotiation.cc:407] Connection header received
0807 00:44:02.170327 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.170329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.170342 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.170412 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:02.171156 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.171701 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.172730 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.172941 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.173022 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.174072 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:02.174074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:02.174074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:02.174082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:02.174105 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:02.174112 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:02.174179 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.174376 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.174426 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:44:02.174906 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.169386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41799 (local address 127.1.116.130:40985)
0807 00:44:02.169697 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.169733 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.170225 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.170492 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.170495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.170503 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:02.171065 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.171070 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.171802 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.171802 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.172618 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.172620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.173058 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.173060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.173176 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.173902 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:44:02.173903 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:02.173930 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:44:02.173952 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:02.174206 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:02.174253 (+ 47us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:02.174265 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.174350 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.174592 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":159,"threads_started":1}
I20260807 00:44:02.175457 28820 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41799
I20260807 00:44:02.175519 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41799
--builtin_ntp_servers=127.1.116.148:40693
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:44:02.175577 28820 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:02.175791 28820 heartbeater.cc:511] Master 127.1.116.190:41799 requested a full tablet report, sending...
I20260807 00:44:02.176211 28300 ts_manager.cc:194] Registered new tserver with Master: 2f17da84a10b475a804f57d47e16a0a7 (127.1.116.130:42355)
I20260807 00:44:02.176789 28300 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.130:40985
W20260807 00:44:02.265161 28828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:02.265407 28828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:02.265434 28828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:02.266660 28828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:02.266701 28828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:02.266737 28828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:44:02.268186 28828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:40693
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:41799
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:02.268435 28828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:02.268769 28828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:02.275111 28834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:02.275116 28836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:02.275616 28839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:02.276222 28828 server_base.cc:1034] running on GCE node
I20260807 00:44:02.276510 28828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:02.276854 28828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:02.278014 28828 hybrid_clock.cc:648] HybridClock initialized: now 1786063442278001 us; error 31 us; skew 500 ppm
I20260807 00:44:02.280089 28828 webserver.cc:460] Webserver started at http://127.1.116.131:43317/ using document root <none> and password file <none>
I20260807 00:44:02.280479 28828 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:02.280566 28828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:02.280743 28828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:02.282274 28828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "36e846c2c674480dba0ab9121f8f1d7a"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "8d95c208f5e99f8845399782544f0163"
server_key_iv: "6d48d15eaf5abf7617fb5450ea128f94"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.282799 28828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "36e846c2c674480dba0ab9121f8f1d7a"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "8d95c208f5e99f8845399782544f0163"
server_key_iv: "6d48d15eaf5abf7617fb5450ea128f94"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.284462 28828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:44:02.285343 28846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.285573 28828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:02.285681 28828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "36e846c2c674480dba0ab9121f8f1d7a"
format_stamp: "Formatted at 2026-08-07 00:44:02 on dist-test-slave-h6n0"
server_key: "8d95c208f5e99f8845399782544f0163"
server_key_iv: "6d48d15eaf5abf7617fb5450ea128f94"
server_key_version: "encryptionkey@0"
I20260807 00:44:02.285795 28828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:02.306773 28828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:02.307147 28828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:02.307317 28828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:02.307597 28828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:02.307996 28828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:02.308034 28828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.308059 28828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:02.308107 28828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:02.315831 28828 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:41273
I20260807 00:44:02.315950 29018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:41273 every 8 connection(s)
I20260807 00:44:02.316313 28828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:44:02.322152 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.317259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51561 (local address 127.1.116.190:41799)
0807 00:44:02.317333 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.317335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.317611 (+ 276us) server_negotiation.cc:407] Connection header received
0807 00:44:02.318183 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.318185 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.318197 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.318282 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:02.319050 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.319571 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.320633 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.320843 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.320916 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.321872 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:02.321873 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:02.321873 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:02.321877 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:02.321892 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:02.321895 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:02.321968 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.322058 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.322090 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:44:02.322335 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.317241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:41799 (local address 127.1.116.131:51561)
0807 00:44:02.317454 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.317499 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.318075 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.318342 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.318345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.318354 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:02.318951 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.318956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.319670 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.319670 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.320532 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.320535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.320937 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.320938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.321053 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.321712 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:44:02.321713 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:02.321750 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:44:02.321775 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:02.321972 (+ 197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:02.321974 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:02.321977 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.322027 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.322114 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":97,"threads_started":1}
I20260807 00:44:02.322789 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 28828
I20260807 00:44:02.322932 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:44:02.323069 29019 heartbeater.cc:347] Connected to a master server at 127.1.116.190:41799
I20260807 00:44:02.323179 29019 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:02.323236 1490 external_mini_cluster.cc:1428] Setting key a7bfe822dfc3b5a26f13bda87e652b49
I20260807 00:44:02.323444 29019 heartbeater.cc:511] Master 127.1.116.190:41799 requested a full tablet report, sending...
I20260807 00:44:02.323900 28300 ts_manager.cc:194] Registered new tserver with Master: 36e846c2c674480dba0ab9121f8f1d7a (127.1.116.131:41273)
I20260807 00:44:02.324368 28300 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.131:51561
I20260807 00:44:02.324438 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:44:02.329234 28398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.326182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57112 (local address 127.1.116.190:41799)
0807 00:44:02.326260 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.326262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.326486 (+ 224us) server_negotiation.cc:407] Connection header received
0807 00:44:02.326573 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.326574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.326587 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.326644 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:02.326814 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.327273 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.328014 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.328201 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.328265 (+ 64us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.328943 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:02.328944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:02.328944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:02.328948 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:02.328962 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:02.328966 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:02.329036 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.329109 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.329120 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:44:02.331113 28300 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:44:02.332816 28300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:44:02.340611 29037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.335638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:42355 (local address 127.0.0.1:45026)
0807 00:44:02.335902 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.335912 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.336043 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.336289 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.336292 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.336294 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:02.336362 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.336364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.337893 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.337894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.339340 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.339343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.340298 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.340300 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.340381 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.340395 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.340469 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.340534 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":41,"thread_start_us":103,"threads_started":1}
I20260807 00:44:02.340994 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.335872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.1.116.130:42355)
0807 00:44:02.336085 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.336090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.336101 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:44:02.336149 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.336151 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.336168 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.336263 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:02.336621 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.337802 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.339444 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.340200 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.340268 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.340341 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.340831 (+ 490us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.340907 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260807 00:44:02.341642 29036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.335604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.1.116.131:41273)
0807 00:44:02.336202 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.336207 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.336216 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:44:02.336258 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.336261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.336278 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.336355 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:02.337898 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.338910 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.340090 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.341272 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.341338 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.341453 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.341519 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.341559 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":87,"threads_started":1}
I20260807 00:44:02.341747 29030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.335400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37857 (local address 127.0.0.1:60386)
0807 00:44:02.335800 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.335813 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.335887 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.336423 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.336424 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.336427 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:02.336504 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.336505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.337894 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.337895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.339340 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.339343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.341487 (+ 2144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.341488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.341550 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.341554 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.341603 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.341644 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"mutex_wait_us":72,"thread_start_us":136,"threads_started":1}
I20260807 00:44:02.341653 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.335365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:41273 (local address 127.0.0.1:54560)
0807 00:44:02.335623 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.335662 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.335730 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.336424 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.336425 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.336430 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:02.336514 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.336516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.339006 (+ 2490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.339007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.339986 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.339989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.341366 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.341367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.341434 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.341438 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.341509 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.341564 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"mutex_wait_us":25,"thread_start_us":99,"threads_started":1}
I20260807 00:44:02.341861 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.335546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60386 (local address 127.1.116.129:37857)
0807 00:44:02.335806 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.335812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.335875 (+ 63us) server_negotiation.cc:407] Connection header received
0807 00:44:02.335973 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.335976 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.335995 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.336415 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:02.336591 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.337771 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.340402 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.341027 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.341131 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.341234 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.341700 (+ 466us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.341757 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":112,"threads_started":1}
I20260807 00:44:02.341830 28708 tablet_service.cc:1467] Processing CreateTablet for tablet 752f82261bee4f289c160eecf8440fa3 (DEFAULT_TABLE table=test-table [id=83ba7ef62b7f4540b197689b79e6af53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:02.342204 28708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752f82261bee4f289c160eecf8440fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:02.342566 28497 tablet_service.cc:1467] Processing CreateTablet for tablet 752f82261bee4f289c160eecf8440fa3 (DEFAULT_TABLE table=test-table [id=83ba7ef62b7f4540b197689b79e6af53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:02.343010 28497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752f82261bee4f289c160eecf8440fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:02.343288 28918 tablet_service.cc:1467] Processing CreateTablet for tablet 752f82261bee4f289c160eecf8440fa3 (DEFAULT_TABLE table=test-table [id=83ba7ef62b7f4540b197689b79e6af53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:02.343621 28918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 752f82261bee4f289c160eecf8440fa3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:02.345284 29040 tablet_bootstrap.cc:492] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Bootstrap starting.
I20260807 00:44:02.345981 29040 tablet_bootstrap.cc:654] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:02.346277 29040 log.cc:824] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:02.347069 29042 tablet_bootstrap.cc:492] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b: Bootstrap starting.
I20260807 00:44:02.347194 29044 tablet_bootstrap.cc:492] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Bootstrap starting.
I20260807 00:44:02.347469 29040 tablet_bootstrap.cc:492] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: No bootstrap required, opened a new log
I20260807 00:44:02.347560 29040 ts_tablet_manager.cc:1397] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:44:02.347793 29042 tablet_bootstrap.cc:654] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:02.347972 29044 tablet_bootstrap.cc:654] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:02.348191 29042 log.cc:824] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:02.348378 29044 log.cc:824] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:02.349145 29042 tablet_bootstrap.cc:492] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b: No bootstrap required, opened a new log
I20260807 00:44:02.349217 29044 tablet_bootstrap.cc:492] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: No bootstrap required, opened a new log
I20260807 00:44:02.349246 29042 ts_tablet_manager.cc:1397] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:44:02.349267 29044 ts_tablet_manager.cc:1397] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:44:02.349882 29040 raft_consensus.cc:348] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.350044 29040 raft_consensus.cc:374] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:02.350087 29040 raft_consensus.cc:729] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f17da84a10b475a804f57d47e16a0a7, State: Initialized, Role: FOLLOWER
I20260807 00:44:02.350193 29040 consensus_queue.cc:260] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.350488 29040 ts_tablet_manager.cc:1428] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:02.350503 28820 heartbeater.cc:503] Master 127.1.116.190:41799 was elected leader, sending a full tablet report...
I20260807 00:44:02.352035 29044 raft_consensus.cc:348] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.352025 29042 raft_consensus.cc:348] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.352207 29044 raft_consensus.cc:374] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:02.352207 29042 raft_consensus.cc:374] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:02.352252 29044 raft_consensus.cc:729] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36e846c2c674480dba0ab9121f8f1d7a, State: Initialized, Role: FOLLOWER
I20260807 00:44:02.352257 29042 raft_consensus.cc:729] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0cbd652de91406985c0f090181be06b, State: Initialized, Role: FOLLOWER
I20260807 00:44:02.352380 29044 consensus_queue.cc:260] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.352509 29042 consensus_queue.cc:260] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.352672 29044 ts_tablet_manager.cc:1428] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260807 00:44:02.352788 29042 ts_tablet_manager.cc:1428] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260807 00:44:02.352774 29019 heartbeater.cc:503] Master 127.1.116.190:41799 was elected leader, sending a full tablet report...
I20260807 00:44:02.352931 28610 heartbeater.cc:503] Master 127.1.116.190:41799 was elected leader, sending a full tablet report...
I20260807 00:44:02.410236 29049 raft_consensus.cc:482] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:44:02.410373 29049 raft_consensus.cc:504] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.410851 29049 leader_election.cc:288] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0cbd652de91406985c0f090181be06b (127.1.116.129:37857), 2f17da84a10b475a804f57d47e16a0a7 (127.1.116.130:42355)
I20260807 00:44:02.414875 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.411119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51113 (local address 127.1.116.129:37857)
0807 00:44:02.411164 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.411166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.411549 (+ 383us) server_negotiation.cc:407] Connection header received
0807 00:44:02.411640 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.411642 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.411655 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.411718 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:02.411886 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.412578 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.413899 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.414460 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.414557 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.414568 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.414739 (+ 171us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.414788 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:44:02.414875 29039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.411128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:60013 (local address 127.1.116.130:42355)
0807 00:44:02.411188 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.411191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.411203 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:44:02.411242 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.411243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.411256 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.411332 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:02.411564 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.412400 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.413820 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.414439 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.414566 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.414575 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.414744 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.414787 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 00:44:02.414880 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.411034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:42355 (local address 127.1.116.131:60013)
0807 00:44:02.411101 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.411113 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.411156 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.411377 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.411379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.411381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:02.411468 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.411470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.412484 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.412485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.413715 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.413717 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.414568 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.414569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.414645 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.414649 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.414714 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.414768 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":12}
I20260807 00:44:02.414898 29051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.411076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37857 (local address 127.1.116.131:51113)
0807 00:44:02.411409 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.411422 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:02.411554 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:02.411740 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:02.411741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:02.411743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:02.411808 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.411809 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.412662 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.412663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.413805 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:02.413807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.414569 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.414570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:02.414645 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.414649 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:02.414715 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:44:02.414766 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"mutex_wait_us":11,"thread_start_us":204,"threads_started":1}
I20260807 00:44:02.415328 28746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752f82261bee4f289c160eecf8440fa3" candidate_uuid: "36e846c2c674480dba0ab9121f8f1d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f17da84a10b475a804f57d47e16a0a7" is_pre_election: true
I20260807 00:44:02.415452 28746 raft_consensus.cc:2454] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36e846c2c674480dba0ab9121f8f1d7a in term 0.
I20260807 00:44:02.415707 28532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752f82261bee4f289c160eecf8440fa3" candidate_uuid: "36e846c2c674480dba0ab9121f8f1d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0cbd652de91406985c0f090181be06b" is_pre_election: true
I20260807 00:44:02.415901 28532 raft_consensus.cc:2454] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36e846c2c674480dba0ab9121f8f1d7a in term 0.
I20260807 00:44:02.416198 28850 leader_election.cc:302] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36e846c2c674480dba0ab9121f8f1d7a, d0cbd652de91406985c0f090181be06b; no voters:
I20260807 00:44:02.416364 29049 raft_consensus.cc:2781] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:44:02.416463 29049 raft_consensus.cc:482] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:44:02.416504 29049 raft_consensus.cc:3037] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:02.417428 29049 raft_consensus.cc:504] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.417606 29049 leader_election.cc:288] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [CANDIDATE]: Term 1 election: Requested vote from peers d0cbd652de91406985c0f090181be06b (127.1.116.129:37857), 2f17da84a10b475a804f57d47e16a0a7 (127.1.116.130:42355)
I20260807 00:44:02.417805 28746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752f82261bee4f289c160eecf8440fa3" candidate_uuid: "36e846c2c674480dba0ab9121f8f1d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f17da84a10b475a804f57d47e16a0a7"
I20260807 00:44:02.417819 28532 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "752f82261bee4f289c160eecf8440fa3" candidate_uuid: "36e846c2c674480dba0ab9121f8f1d7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0cbd652de91406985c0f090181be06b"
I20260807 00:44:02.417896 28532 raft_consensus.cc:3037] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:02.417917 28746 raft_consensus.cc:3037] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:02.418781 28746 raft_consensus.cc:2454] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36e846c2c674480dba0ab9121f8f1d7a in term 1.
I20260807 00:44:02.418821 28532 raft_consensus.cc:2454] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36e846c2c674480dba0ab9121f8f1d7a in term 1.
I20260807 00:44:02.419000 28850 leader_election.cc:302] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36e846c2c674480dba0ab9121f8f1d7a, d0cbd652de91406985c0f090181be06b; no voters:
W20260807 00:44:02.419080 28825 tablet.cc:2369] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:44:02.419111 29049 raft_consensus.cc:2781] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:02.419271 29049 raft_consensus.cc:686] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 1 LEADER]: Becoming Leader. State: Replica: 36e846c2c674480dba0ab9121f8f1d7a, State: Running, Role: LEADER
I20260807 00:44:02.419394 29049 consensus_queue.cc:237] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } }
I20260807 00:44:02.420310 28300 catalog_manager.cc:5495] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 36e846c2c674480dba0ab9121f8f1d7a (127.1.116.131). New cstate: current_term: 1 leader_uuid: "36e846c2c674480dba0ab9121f8f1d7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36e846c2c674480dba0ab9121f8f1d7a" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 } health_report { overall_health: UNKNOWN } } }
I20260807 00:44:02.433797 29036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.431484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54566 (local address 127.1.116.131:41273)
0807 00:44:02.431543 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:02.431545 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:02.431555 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:44:02.431599 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:02.431600 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:02.431626 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:02.431701 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:02.431837 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.432285 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.433013 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:02.433204 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:02.433275 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:02.433438 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:02.433533 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:02.433633 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:02.433696 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 00:44:02.433704 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 00:44:02.435842 28532 raft_consensus.cc:1261] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 1 FOLLOWER]: Refusing update from remote peer 36e846c2c674480dba0ab9121f8f1d7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:02.436224 28746 raft_consensus.cc:1261] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 1 FOLLOWER]: Refusing update from remote peer 36e846c2c674480dba0ab9121f8f1d7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:02.436378 29052 consensus_queue.cc:1035] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0cbd652de91406985c0f090181be06b" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 37857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:02.436573 29049 consensus_queue.cc:1035] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f17da84a10b475a804f57d47e16a0a7" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 42355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:02.439121 29056 mvcc.cc:205] Tried to move back new op lower bound from 7315715860214149120 to 7315715860150747136. Current Snapshot: MvccSnapshot[applied={T|T < 7315715860214149120}]
I20260807 00:44:02.440269 29059 mvcc.cc:205] Tried to move back new op lower bound from 7315715860214149120 to 7315715860150747136. Current Snapshot: MvccSnapshot[applied={T|T < 7315715860214149120}]
I20260807 00:44:02.441883 28300 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:02.442006 28300 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:02.443039 28300 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 32b39005f5824650bc9280451065eeaa: Sending DeleteTablet for 3 replicas of tablet 752f82261bee4f289c160eecf8440fa3
I20260807 00:44:02.443421 28497 tablet_service.cc:1514] Processing DeleteTablet for tablet 752f82261bee4f289c160eecf8440fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:02 UTC) from {username='slave'} at 127.0.0.1:60386
I20260807 00:44:02.443542 28918 tablet_service.cc:1514] Processing DeleteTablet for tablet 752f82261bee4f289c160eecf8440fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:02 UTC) from {username='slave'} at 127.0.0.1:54560
I20260807 00:44:02.443725 29075 tablet_replica.cc:330] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b: stopping tablet replica
I20260807 00:44:02.443797 28708 tablet_service.cc:1514] Processing DeleteTablet for tablet 752f82261bee4f289c160eecf8440fa3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:02 UTC) from {username='slave'} at 127.0.0.1:45026
I20260807 00:44:02.443881 29075 raft_consensus.cc:2229] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:44:02.443796 29076 tablet_replica.cc:330] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: stopping tablet replica
I20260807 00:44:02.444018 29075 raft_consensus.cc:2258] T 752f82261bee4f289c160eecf8440fa3 P d0cbd652de91406985c0f090181be06b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:02.444119 29076 raft_consensus.cc:2229] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:44:02.444270 29076 raft_consensus.cc:2258] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:02.444392 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 28401
I20260807 00:44:02.451200 29076 ts_tablet_manager.cc:1905] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:44:02.452600 29078 tablet_replica.cc:330] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: stopping tablet replica
I20260807 00:44:02.452786 29078 raft_consensus.cc:2229] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:44:02.452932 29078 raft_consensus.cc:2258] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:02.453409 29076 ts_tablet_manager.cc:1918] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:44:02.453483 29078 ts_tablet_manager.cc:1905] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:44:02.453743 29076 log.cc:1196] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/752f82261bee4f289c160eecf8440fa3
I20260807 00:44:02.454202 29076 ts_tablet_manager.cc:1939] T 752f82261bee4f289c160eecf8440fa3 P 36e846c2c674480dba0ab9121f8f1d7a: Deleting consensus metadata
I20260807 00:44:02.454720 28277 catalog_manager.cc:4841] TS 36e846c2c674480dba0ab9121f8f1d7a (127.1.116.131:41273): tablet 752f82261bee4f289c160eecf8440fa3 (table test-table [id=83ba7ef62b7f4540b197689b79e6af53]) successfully deleted
I20260807 00:44:02.455822 29078 ts_tablet_manager.cc:1918] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:44:02.455914 29078 log.cc:1196] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/752f82261bee4f289c160eecf8440fa3
I20260807 00:44:02.456208 29078 ts_tablet_manager.cc:1939] T 752f82261bee4f289c160eecf8440fa3 P 2f17da84a10b475a804f57d47e16a0a7: Deleting consensus metadata
I20260807 00:44:02.456733 28277 catalog_manager.cc:4841] TS 2f17da84a10b475a804f57d47e16a0a7 (127.1.116.130:42355): tablet 752f82261bee4f289c160eecf8440fa3 (table test-table [id=83ba7ef62b7f4540b197689b79e6af53]) successfully deleted
W20260807 00:44:02.459770 28281 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:37857 (error 108)
I20260807 00:44:02.460147 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 28615
I20260807 00:44:02.460326 29033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:02.460169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:37857 (local address 127.0.0.1:60388)
0807 00:44:02.460244 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:02.460293 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37857: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 00:44:02.483603 28281 catalog_manager.cc:4584] TS d0cbd652de91406985c0f090181be06b (127.1.116.129:37857): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 752f82261bee4f289c160eecf8440fa3: Network error: Client connection negotiation failed: client connection to 127.1.116.129:37857: connect: Connection refused (error 111)
I20260807 00:44:02.490765 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 28828
I20260807 00:44:02.501796 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 28260
2026-08-07T00:44:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (806 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:02 dist-test-slave-h6n0 krb5kdc[29085](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:02 dist-test-slave-h6n0 krb5kdc[29085](info): set up 2 sockets
Aug 07 00:44:02 dist-test-slave-h6n0 krb5kdc[29085](info): commencing operation
krb5kdc: starting...
W20260807 00:44:03.755020 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:03 dist-test-slave-h6n0 krb5kdc[29085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:03.889899 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39619
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:03.979370 29101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:03.979629 29101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:03.979667 29101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:03.980795 29101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:03.980865 29101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:03.980892 29101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:03.980923 29101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:03.980948 29101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:03.982379 29101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:03.982703 29101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:03.982925 29101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:03.988145 29108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:03.988142 29106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:03.988145 29112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:03.988524 29101 server_base.cc:1034] running on GCE node
I20260807 00:44:03.988775 29101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:03.989076 29101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:03.990267 29101 hybrid_clock.cc:648] HybridClock initialized: now 1786063443990247 us; error 36 us; skew 500 ppm
Aug 07 00:44:03 dist-test-slave-h6n0 krb5kdc[29085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063443, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:03.993009 29101 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:03.993506 29101 webserver.cc:460] Webserver started at http://127.1.116.190:37541/ using document root <none> and password file <none>
I20260807 00:44:03.993733 29101 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:03.993805 29101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:03.993955 29101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:03.995052 29101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "9f9d9dc551b04fc48048ec6c946dad74"
format_stamp: "Formatted at 2026-08-07 00:44:03 on dist-test-slave-h6n0"
server_key: "584dcc3d526594aa13baf488754bd800"
server_key_iv: "83b6d3cd85b05932d9231de3c98db096"
server_key_version: "encryptionkey@0"
I20260807 00:44:03.995409 29101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "9f9d9dc551b04fc48048ec6c946dad74"
format_stamp: "Formatted at 2026-08-07 00:44:03 on dist-test-slave-h6n0"
server_key: "584dcc3d526594aa13baf488754bd800"
server_key_iv: "83b6d3cd85b05932d9231de3c98db096"
server_key_version: "encryptionkey@0"
I20260807 00:44:03.996838 29101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:44:03.997705 29121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:03.997995 29101 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:03.998108 29101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "9f9d9dc551b04fc48048ec6c946dad74"
format_stamp: "Formatted at 2026-08-07 00:44:03 on dist-test-slave-h6n0"
server_key: "584dcc3d526594aa13baf488754bd800"
server_key_iv: "83b6d3cd85b05932d9231de3c98db096"
server_key_version: "encryptionkey@0"
I20260807 00:44:03.998193 29101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:04.012588 29101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:04.014439 29101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:04.014621 29101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:04.019948 29101 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:39619
I20260807 00:44:04.020109 29201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:39619 every 8 connection(s)
I20260807 00:44:04.020397 29101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:04.021454 29202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:04.024345 29202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74: Bootstrap starting.
I20260807 00:44:04.025161 29202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:04.025529 29202 log.cc:824] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:04.026439 29202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74: No bootstrap required, opened a new log
I20260807 00:44:04.026479 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 29101
I20260807 00:44:04.026648 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:04.026906 1490 external_mini_cluster.cc:1428] Setting key 7267e617784fbe803990dea25f61f22a
I20260807 00:44:04.028962 29202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9d9dc551b04fc48048ec6c946dad74" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39619 } }
I20260807 00:44:04.029178 29202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:04.029253 29202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f9d9dc551b04fc48048ec6c946dad74, State: Initialized, Role: FOLLOWER
I20260807 00:44:04.029412 29202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9d9dc551b04fc48048ec6c946dad74" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39619 } }
I20260807 00:44:04.029515 29202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:04.029551 29202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:04.029613 29202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:04.030881 29202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9d9dc551b04fc48048ec6c946dad74" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39619 } }
I20260807 00:44:04.030980 29202 leader_election.cc:302] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f9d9dc551b04fc48048ec6c946dad74; no voters:
I20260807 00:44:04.031203 29202 leader_election.cc:288] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:04.031266 29209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:04.031467 29209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [term 1 LEADER]: Becoming Leader. State: Replica: 9f9d9dc551b04fc48048ec6c946dad74, State: Running, Role: LEADER
I20260807 00:44:04.031533 29202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:04.031600 29209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9d9dc551b04fc48048ec6c946dad74" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39619 } }
I20260807 00:44:04.032023 29209 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f9d9dc551b04fc48048ec6c946dad74. Latest consensus state: current_term: 1 leader_uuid: "9f9d9dc551b04fc48048ec6c946dad74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9d9dc551b04fc48048ec6c946dad74" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39619 } } }
I20260807 00:44:04.032120 29209 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:04.032279 29210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f9d9dc551b04fc48048ec6c946dad74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f9d9dc551b04fc48048ec6c946dad74" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39619 } } }
I20260807 00:44:04.032344 29210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:04.032465 29219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:04.033069 29219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 00:44:04 dist-test-slave-h6n0 krb5kdc[29085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:04.036782 29219 catalog_manager.cc:1347] Generated new cluster ID: 4037b308c6244c48b87aca8b27a39331
I20260807 00:44:04.036841 29219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:04.046021 29219 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:04.046454 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.027521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35674 (local address 127.1.116.190:39619)
0807 00:44:04.027735 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.027742 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.027770 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:44:04.028972 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.028980 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.029073 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.029254 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:04.030227 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.031379 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.032533 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.033123 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.033544 (+ 421us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.036216 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:04.036221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:04.036221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:04.036235 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:04.044364 (+ 8129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.044839 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.044843 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.044843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.044964 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.045212 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.045213 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.045213 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.045513 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:04.045738 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.045856 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.046023 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260807 00:44:04.046914 29219 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:04.063556 29219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74: Generated new TSK 0
I20260807 00:44:04.063830 29219 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.116.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
Entry for principal kudu/127.1.116.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab.
I20260807 00:44:04.099247 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39619
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:04.203217 29243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:04.203457 29243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:04.203505 29243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:04.204731 29243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:04.204775 29243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:04.204802 29243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:44:04.206315 29243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.129.keytab
--principal=kudu/127.1.116.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:04.206616 29243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:04.206898 29243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:04.212738 29251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:04.212790 29252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:04.212922 29254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:04.213224 29243 server_base.cc:1034] running on GCE node
I20260807 00:44:04.213482 29243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:04.213761 29243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:04.214972 29243 hybrid_clock.cc:648] HybridClock initialized: now 1786063444214933 us; error 38 us; skew 500 ppm
Aug 07 00:44:04 dist-test-slave-h6n0 krb5kdc[29085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:04.218169 29243 init.cc:377] Logged in from keytab as kudu/127.1.116.129@KRBTEST.COM (short username kudu)
I20260807 00:44:04.218716 29243 webserver.cc:460] Webserver started at http://127.1.116.129:45763/ using document root <none> and password file <none>
I20260807 00:44:04.218978 29243 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:04.219050 29243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:04.219210 29243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:04.220472 29243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "7857a79570b1499f98595c814d9f76c5"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "9b4c2c415fd36d8d016cc1e096d0d727"
server_key_iv: "626f1ab25b1dd2511e768f328e56ce7d"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.220867 29243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "7857a79570b1499f98595c814d9f76c5"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "9b4c2c415fd36d8d016cc1e096d0d727"
server_key_iv: "626f1ab25b1dd2511e768f328e56ce7d"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.222401 29243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:44:04.223454 29263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.223784 29243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:04.223943 29243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "7857a79570b1499f98595c814d9f76c5"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "9b4c2c415fd36d8d016cc1e096d0d727"
server_key_iv: "626f1ab25b1dd2511e768f328e56ce7d"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.224025 29243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:04.245522 29243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:04.247668 29243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:04.247833 29243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:04.248061 29243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:04.248397 29243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:04.248456 29243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.248488 29243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:04.248517 29243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.255290 29243 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:41153
I20260807 00:44:04.255407 29446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:41153 every 8 connection(s)
I20260807 00:44:04.255697 29243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:44:04.257033 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 29243
I20260807 00:44:04.257159 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:44:04.257373 1490 external_mini_cluster.cc:1428] Setting key b166066b75f947a72b46ebcabcfafd0d
Aug 07 00:44:04 dist-test-slave-h6n0 krb5kdc[29085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.129@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.130@KRBTEST.COM; defaulting to no policy
I20260807 00:44:04.267620 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.256690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:51495 (local address 127.1.116.190:39619)
0807 00:44:04.256759 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.256761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.257024 (+ 263us) server_negotiation.cc:407] Connection header received
0807 00:44:04.257852 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.257853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.257863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.257925 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:04.259183 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.259937 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.261541 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.261741 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.261815 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.264473 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:04.264476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:04.264476 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:04.264484 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:04.266069 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.266510 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.266512 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.266512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.266612 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.266886 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.266887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.266887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.267018 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:04.267098 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.267405 (+ 307us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.267507 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 00:44:04.267812 29453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.256651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39619 (local address 127.1.116.129:51495)
0807 00:44:04.256878 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.256925 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.257717 (+ 792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.258023 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.258027 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.258823 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:04.259056 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.259060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.260061 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.260063 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.261437 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.261440 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.261835 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.261837 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.261985 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.262627 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:04.262643 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:04.264333 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:04.266179 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:04.266187 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:04.266189 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:04.266428 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:04.266696 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:04.266697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:04.266697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:04.266810 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:04.267087 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:04.267095 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:04.267266 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.267382 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.267562 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":128,"threads_started":1}
I20260807 00:44:04.268484 29447 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39619
I20260807 00:44:04.268608 29447 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:04.268826 29447 heartbeater.cc:511] Master 127.1.116.190:39619 requested a full tablet report, sending...
I20260807 00:44:04.269558 29147 ts_manager.cc:194] Registered new tserver with Master: 7857a79570b1499f98595c814d9f76c5 (127.1.116.129:41153)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.130@KRBTEST.COM" created.
I20260807 00:44:04.270365 29147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.129@KRBTEST.COM'} at 127.1.116.129:51495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
Entry for principal kudu/127.1.116.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab.
I20260807 00:44:04.283416 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39619
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:04.372877 29458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:04.373077 29458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:04.373099 29458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:04.374126 29458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:04.374164 29458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:04.374189 29458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:44:04.375598 29458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.130.keytab
--principal=kudu/127.1.116.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:04.375789 29458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:04.375949 29458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:04.381055 29470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:04.381172 29469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:04.381296 29458 server_base.cc:1034] running on GCE node
W20260807 00:44:04.381373 29472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:04.381646 29458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:04.381974 29458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:04.383160 29458 hybrid_clock.cc:648] HybridClock initialized: now 1786063444383129 us; error 49 us; skew 500 ppm
Aug 07 00:44:04 dist-test-slave-h6n0 krb5kdc[29085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:04.386163 29458 init.cc:377] Logged in from keytab as kudu/127.1.116.130@KRBTEST.COM (short username kudu)
I20260807 00:44:04.386756 29458 webserver.cc:460] Webserver started at http://127.1.116.130:38985/ using document root <none> and password file <none>
I20260807 00:44:04.387004 29458 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:04.387075 29458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:04.387247 29458 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:04.388449 29458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "7f74746f0f4e4966a31ea61f6ba69c77"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "5ada6d0846a672b87280c21666f62c33"
server_key_iv: "bf30f64def036c4dae7578063bc97cd6"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.388855 29458 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "7f74746f0f4e4966a31ea61f6ba69c77"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "5ada6d0846a672b87280c21666f62c33"
server_key_iv: "bf30f64def036c4dae7578063bc97cd6"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.390399 29458 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:04.391546 29481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.391871 29458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:04.392004 29458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "7f74746f0f4e4966a31ea61f6ba69c77"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "5ada6d0846a672b87280c21666f62c33"
server_key_iv: "bf30f64def036c4dae7578063bc97cd6"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.392102 29458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:04.401897 29458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:04.404064 29458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:04.404184 29458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:04.404373 29458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:04.404719 29458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:04.404762 29458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.404784 29458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:04.404799 29458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.411441 29458 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:35551
I20260807 00:44:04.411531 29642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:35551 every 8 connection(s)
I20260807 00:44:04.411852 29458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
Aug 07 00:44:04 dist-test-slave-h6n0 krb5kdc[29085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.130@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:04.419842 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 29458
I20260807 00:44:04.419976 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:44:04.420217 1490 external_mini_cluster.cc:1428] Setting key 70f047226c8c589258aae83c4cdc0619
I20260807 00:44:04.423244 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.412795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:46211 (local address 127.1.116.190:39619)
0807 00:44:04.412894 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.412897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.413100 (+ 203us) server_negotiation.cc:407] Connection header received
0807 00:44:04.413817 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.413818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.413833 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.413900 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:04.415100 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.415778 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.416874 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.417106 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.417200 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.419620 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:04.419622 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:04.419622 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:04.419638 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:04.421507 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.422084 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.422085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.422085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.422166 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.422493 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.422493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.422493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.422683 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:04.422744 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.423103 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.423179 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:44:04.423687 29646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.412672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39619 (local address 127.1.116.130:46211)
0807 00:44:04.412928 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.412989 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.413686 (+ 697us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.413982 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.413987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.414743 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:04.414978 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.414984 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.415892 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.415893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.416772 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.416774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.417210 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.417212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.417356 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.417974 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:04.417983 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:04.419518 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:04.421605 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:04.421613 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:04.421615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:04.421991 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:04.422247 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:04.422248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:04.422249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:04.422410 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:04.422760 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:04.422766 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:04.422951 (+ 185us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.423080 (+ 129us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.423294 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":116,"threads_started":1}
I20260807 00:44:04.424316 29643 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39619
I20260807 00:44:04.424438 29643 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:04.424657 29643 heartbeater.cc:511] Master 127.1.116.190:39619 requested a full tablet report, sending...
I20260807 00:44:04.425112 29147 ts_manager.cc:194] Registered new tserver with Master: 7f74746f0f4e4966a31ea61f6ba69c77 (127.1.116.130:35551)
I20260807 00:44:04.425520 29147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.130@KRBTEST.COM'} at 127.1.116.130:46211
WARNING: no policy specified for kudu/127.1.116.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
Entry for principal kudu/127.1.116.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab.
I20260807 00:44:04.451241 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39619
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:35549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:04.550398 29650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:04.550714 29650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:04.550743 29650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:04.551915 29650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:04.551965 29650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:04.551992 29650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:44:04.553517 29650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:35549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.131.keytab
--principal=kudu/127.1.116.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:39619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29650
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:04.553795 29650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:04.554054 29650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:04.559808 29655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:04.559814 29657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:04.559845 29660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:04.560096 29650 server_base.cc:1034] running on GCE node
I20260807 00:44:04.560436 29650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:04.560743 29650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:04.561949 29650 hybrid_clock.cc:648] HybridClock initialized: now 1786063444561926 us; error 47 us; skew 500 ppm
Aug 07 00:44:04 dist-test-slave-h6n0 krb5kdc[29085](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:04.564865 29650 init.cc:377] Logged in from keytab as kudu/127.1.116.131@KRBTEST.COM (short username kudu)
I20260807 00:44:04.565407 29650 webserver.cc:460] Webserver started at http://127.1.116.131:38797/ using document root <none> and password file <none>
I20260807 00:44:04.565611 29650 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:04.565657 29650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:04.565785 29650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:04.566921 29650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "7175925677f148b0ae4153a5ea1d5fdf"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "2f379f1f08aa2e2a1802e8ee7c26a60d"
server_key_iv: "8dbcb7591118d58c5a651fbe8b46717e"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.567273 29650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "7175925677f148b0ae4153a5ea1d5fdf"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "2f379f1f08aa2e2a1802e8ee7c26a60d"
server_key_iv: "8dbcb7591118d58c5a651fbe8b46717e"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.568676 29650 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260807 00:44:04.569489 29674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.569770 29650 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:04.569885 29650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "7175925677f148b0ae4153a5ea1d5fdf"
format_stamp: "Formatted at 2026-08-07 00:44:04 on dist-test-slave-h6n0"
server_key: "2f379f1f08aa2e2a1802e8ee7c26a60d"
server_key_iv: "8dbcb7591118d58c5a651fbe8b46717e"
server_key_version: "encryptionkey@0"
I20260807 00:44:04.569964 29650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:04.579511 29650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:04.581993 29650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:04.582124 29650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:04.582397 29650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:04.582829 29650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:04.582871 29650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.582894 29650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:04.582907 29650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:04.590098 29650 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:41491
I20260807 00:44:04.590196 29848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:41491 every 8 connection(s)
I20260807 00:44:04.590648 29650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
Aug 07 00:44:04 dist-test-slave-h6n0 krb5kdc[29085](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.131@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:04.597886 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 29650
I20260807 00:44:04.598014 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:44:04.598258 1490 external_mini_cluster.cc:1428] Setting key 051db535228004003228c2c4560c8c27
I20260807 00:44:04.601547 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.591598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:50247 (local address 127.1.116.190:39619)
0807 00:44:04.591665 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.591668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.591886 (+ 218us) server_negotiation.cc:407] Connection header received
0807 00:44:04.592373 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.592375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.592390 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.592454 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:04.593391 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.593949 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.595437 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.595643 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.595706 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.598201 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:04.598203 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:04.598203 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:04.598211 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:04.600058 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.600550 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.600551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.600551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.600623 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.600951 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.600952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.600952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.601073 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:04.601132 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.601412 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.601483 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 00:44:04.601783 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.591580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39619 (local address 127.1.116.131:50247)
0807 00:44:04.591764 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.591801 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.592267 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.592507 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.592510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.593102 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:04.593294 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.593299 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.594121 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.594123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.595333 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.595339 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.595739 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.595741 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.595848 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.596350 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:04.596359 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:04.598026 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:04.600143 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:04.600151 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:04.600153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:04.600452 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:04.600714 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:04.600715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:04.600715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:04.600873 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:04.601140 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:04.601145 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:04.601277 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.601368 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.601513 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260807 00:44:04.602387 29849 heartbeater.cc:347] Connected to a master server at 127.1.116.190:39619
I20260807 00:44:04.602491 29849 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:04.602708 29849 heartbeater.cc:511] Master 127.1.116.190:39619 requested a full tablet report, sending...
I20260807 00:44:04.603134 29147 ts_manager.cc:194] Registered new tserver with Master: 7175925677f148b0ae4153a5ea1d5fdf (127.1.116.131:41491)
I20260807 00:44:04.603521 29147 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.116.131@KRBTEST.COM'} at 127.1.116.131:50247
I20260807 00:44:04.610522 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:44:04.618384 29208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.612140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35684 (local address 127.1.116.190:39619)
0807 00:44:04.612215 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.612217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.612348 (+ 131us) server_negotiation.cc:407] Connection header received
0807 00:44:04.612434 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.612435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.612446 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.612493 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:04.613085 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.613642 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.614402 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.614703 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.614780 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.615970 (+ 1190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:04.615971 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:04.615971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:04.615979 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:04.617150 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.617513 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.617515 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.617515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.617570 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:04.617784 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:04.617785 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:04.617785 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:04.617943 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:04.618060 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.618162 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.618268 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:44:04.620059 29147 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35684:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:44:04.622074 29147 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:44:04.629761 29867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.625203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:41153 (local address 127.0.0.1:59330)
0807 00:44:04.625433 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.625444 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.625506 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.626017 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.626018 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.626021 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:04.626081 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.626083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.627231 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.627233 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.628524 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.628527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.629449 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.629451 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.629520 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.629532 (+ 12us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.629600 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.629652 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":139,"thread_start_us":101,"threads_started":1}
I20260807 00:44:04.630045 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.625540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59330 (local address 127.1.116.129:41153)
0807 00:44:04.625750 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.625755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.625766 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:44:04.625815 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.625817 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.625837 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.626020 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:04.626173 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.627120 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.628643 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.629347 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.629433 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.629496 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.629638 (+ 142us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.629731 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":104,"threads_started":1}
I20260807 00:44:04.630366 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.625006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.1.116.131:41491)
0807 00:44:04.625408 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.625412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.625463 (+ 51us) server_negotiation.cc:407] Connection header received
0807 00:44:04.625847 (+ 384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.625850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.625864 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.625917 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:04.626141 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.627298 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.628884 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.629606 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.629696 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.629751 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.630144 (+ 393us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.630216 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":123,"threads_started":1}
I20260807 00:44:04.630631 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.624917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:41491 (local address 127.0.0.1:44156)
0807 00:44:04.625383 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.625396 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.625725 (+ 329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.626011 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.626012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.626014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:04.626060 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.626062 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.627434 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.627435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.628783 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.628785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.629721 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.629722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.629797 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.629801 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.630498 (+ 697us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.630569 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":54,"thread_start_us":79,"threads_started":1}
I20260807 00:44:04.631107 29866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.624917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:35551 (local address 127.0.0.1:36986)
0807 00:44:04.625232 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.625273 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.625627 (+ 354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.625956 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.625959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.625963 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:04.626039 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.626042 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.627435 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.627435 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.628781 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.628785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.630824 (+ 2039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.630825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.630899 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.630905 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.630990 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.631052 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":180,"thread_start_us":98,"threads_started":1}
I20260807 00:44:04.631032 29341 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff6d2799c9842dda69f3f1907304ff6 (DEFAULT_TABLE table=test-table [id=4165a2ecba0348cca225326803da82ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:04.631290 29752 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff6d2799c9842dda69f3f1907304ff6 (DEFAULT_TABLE table=test-table [id=4165a2ecba0348cca225326803da82ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:04.631448 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.625025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36986 (local address 127.1.116.130:35551)
0807 00:44:04.625287 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.625294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.625373 (+ 79us) server_negotiation.cc:407] Connection header received
0807 00:44:04.625734 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.625737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.625755 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.625813 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:04.626134 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.627298 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.629805 (+ 2507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.630646 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.630724 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.630783 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.631302 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.631359 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":101,"threads_started":1}
I20260807 00:44:04.631435 29341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff6d2799c9842dda69f3f1907304ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:04.631630 29752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff6d2799c9842dda69f3f1907304ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:04.632170 29550 tablet_service.cc:1467] Processing CreateTablet for tablet 7ff6d2799c9842dda69f3f1907304ff6 (DEFAULT_TABLE table=test-table [id=4165a2ecba0348cca225326803da82ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:04.632556 29550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7ff6d2799c9842dda69f3f1907304ff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:04.635113 29874 tablet_bootstrap.cc:492] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Bootstrap starting.
I20260807 00:44:04.636008 29874 tablet_bootstrap.cc:654] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:04.636309 29877 tablet_bootstrap.cc:492] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: Bootstrap starting.
I20260807 00:44:04.636636 29874 log.cc:824] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:04.637017 29877 tablet_bootstrap.cc:654] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:04.637024 29872 tablet_bootstrap.cc:492] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5: Bootstrap starting.
I20260807 00:44:04.637521 29874 tablet_bootstrap.cc:492] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: No bootstrap required, opened a new log
I20260807 00:44:04.637543 29877 log.cc:824] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:04.637605 29874 ts_tablet_manager.cc:1397] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:44:04.637791 29872 tablet_bootstrap.cc:654] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:04.638293 29877 tablet_bootstrap.cc:492] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: No bootstrap required, opened a new log
I20260807 00:44:04.638382 29877 ts_tablet_manager.cc:1397] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:44:04.638410 29872 log.cc:824] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:04.639357 29872 tablet_bootstrap.cc:492] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5: No bootstrap required, opened a new log
I20260807 00:44:04.639448 29872 ts_tablet_manager.cc:1397] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:44:04.639863 29874 raft_consensus.cc:348] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.640051 29874 raft_consensus.cc:374] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:04.640096 29874 raft_consensus.cc:729] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7175925677f148b0ae4153a5ea1d5fdf, State: Initialized, Role: FOLLOWER
I20260807 00:44:04.640250 29874 consensus_queue.cc:260] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.640588 29874 ts_tablet_manager.cc:1428] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:04.640604 29849 heartbeater.cc:503] Master 127.1.116.190:39619 was elected leader, sending a full tablet report...
I20260807 00:44:04.640569 29877 raft_consensus.cc:348] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.640764 29877 raft_consensus.cc:374] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:04.640806 29877 raft_consensus.cc:729] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f74746f0f4e4966a31ea61f6ba69c77, State: Initialized, Role: FOLLOWER
I20260807 00:44:04.640947 29877 consensus_queue.cc:260] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.641247 29877 ts_tablet_manager.cc:1428] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:04.641288 29643 heartbeater.cc:503] Master 127.1.116.190:39619 was elected leader, sending a full tablet report...
I20260807 00:44:04.641440 29872 raft_consensus.cc:348] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.641639 29872 raft_consensus.cc:374] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:04.641692 29872 raft_consensus.cc:729] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7857a79570b1499f98595c814d9f76c5, State: Initialized, Role: FOLLOWER
I20260807 00:44:04.641806 29872 consensus_queue.cc:260] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.642104 29872 ts_tablet_manager.cc:1428] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:04.642258 29447 heartbeater.cc:503] Master 127.1.116.190:39619 was elected leader, sending a full tablet report...
W20260807 00:44:04.662587 29645 tablet.cc:2369] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:44:04.756470 29451 tablet.cc:2369] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:44:04.841325 29853 tablet.cc:2369] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:44:04.937307 29885 raft_consensus.cc:482] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:44:04.937454 29885 raft_consensus.cc:504] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.937867 29885 leader_election.cc:288] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f74746f0f4e4966a31ea61f6ba69c77 (127.1.116.130:35551), 7857a79570b1499f98595c814d9f76c5 (127.1.116.129:41153)
I20260807 00:44:04.942436 29871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.938511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:59977 (local address 127.1.116.129:41153)
0807 00:44:04.938668 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.938670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.938812 (+ 142us) server_negotiation.cc:407] Connection header received
0807 00:44:04.938907 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.938908 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.938923 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.939036 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:04.939246 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.939904 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.941373 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.941951 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.942048 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.942057 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.942318 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.942362 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260807 00:44:04.942436 29869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.938357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:36565 (local address 127.1.116.130:35551)
0807 00:44:04.938440 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.938442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.938456 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:44:04.938569 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.938570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.938581 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.938635 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:04.938909 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.939684 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.941373 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.941951 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.942048 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.942056 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.942320 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.942360 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:44:04.942461 29854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.938250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:35551 (local address 127.1.116.131:36565)
0807 00:44:04.938366 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.938385 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.938469 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.938679 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.938680 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.938683 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:04.938785 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.938788 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.939817 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.939818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.941240 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.941243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.942078 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.942079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.942221 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.942223 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.942292 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.942351 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":42}
I20260807 00:44:04.942467 29888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.938413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:41153 (local address 127.1.116.131:59977)
0807 00:44:04.938678 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:04.938694 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:04.938773 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:04.939072 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:04.939072 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:04.939074 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:04.939146 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.939149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.940000 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.940001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.941240 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:04.941243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.942078 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.942079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:04.942214 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.942221 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:44:04.942292 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:44:04.942351 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":50,"thread_start_us":191,"threads_started":1}
I20260807 00:44:04.942909 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff6d2799c9842dda69f3f1907304ff6" candidate_uuid: "7175925677f148b0ae4153a5ea1d5fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" is_pre_election: true
I20260807 00:44:04.943082 29574 raft_consensus.cc:2454] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7175925677f148b0ae4153a5ea1d5fdf in term 0.
I20260807 00:44:04.943028 29377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff6d2799c9842dda69f3f1907304ff6" candidate_uuid: "7175925677f148b0ae4153a5ea1d5fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7857a79570b1499f98595c814d9f76c5" is_pre_election: true
I20260807 00:44:04.943244 29377 raft_consensus.cc:2454] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7175925677f148b0ae4153a5ea1d5fdf in term 0.
I20260807 00:44:04.943329 29680 leader_election.cc:302] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7175925677f148b0ae4153a5ea1d5fdf, 7f74746f0f4e4966a31ea61f6ba69c77; no voters:
I20260807 00:44:04.943518 29885 raft_consensus.cc:2781] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:44:04.943598 29885 raft_consensus.cc:482] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:44:04.943622 29885 raft_consensus.cc:3037] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:04.944523 29885 raft_consensus.cc:504] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.944653 29885 leader_election.cc:288] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [CANDIDATE]: Term 1 election: Requested vote from peers 7f74746f0f4e4966a31ea61f6ba69c77 (127.1.116.130:35551), 7857a79570b1499f98595c814d9f76c5 (127.1.116.129:41153)
I20260807 00:44:04.944936 29574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff6d2799c9842dda69f3f1907304ff6" candidate_uuid: "7175925677f148b0ae4153a5ea1d5fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f74746f0f4e4966a31ea61f6ba69c77"
I20260807 00:44:04.945024 29377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7ff6d2799c9842dda69f3f1907304ff6" candidate_uuid: "7175925677f148b0ae4153a5ea1d5fdf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7857a79570b1499f98595c814d9f76c5"
I20260807 00:44:04.945065 29574 raft_consensus.cc:3037] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:04.945156 29377 raft_consensus.cc:3037] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:04.945973 29574 raft_consensus.cc:2454] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7175925677f148b0ae4153a5ea1d5fdf in term 1.
I20260807 00:44:04.946002 29377 raft_consensus.cc:2454] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7175925677f148b0ae4153a5ea1d5fdf in term 1.
I20260807 00:44:04.946188 29680 leader_election.cc:302] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7175925677f148b0ae4153a5ea1d5fdf, 7857a79570b1499f98595c814d9f76c5; no voters:
I20260807 00:44:04.946321 29885 raft_consensus.cc:2781] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:04.946611 29885 raft_consensus.cc:686] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 1 LEADER]: Becoming Leader. State: Replica: 7175925677f148b0ae4153a5ea1d5fdf, State: Running, Role: LEADER
I20260807 00:44:04.946738 29885 consensus_queue.cc:237] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } }
I20260807 00:44:04.947798 29145 catalog_manager.cc:5495] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf reported cstate change: term changed from 0 to 1, leader changed from <none> to 7175925677f148b0ae4153a5ea1d5fdf (127.1.116.131). New cstate: current_term: 1 leader_uuid: "7175925677f148b0ae4153a5ea1d5fdf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7175925677f148b0ae4153a5ea1d5fdf" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 41491 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 } health_report { overall_health: UNKNOWN } } }
I20260807 00:44:04.985819 29868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:04.983380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.1.116.131:41491)
0807 00:44:04.983460 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:04.983463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:04.983475 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:44:04.983533 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:04.983534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:04.983551 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:04.983611 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:04.983772 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.984240 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.985084 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:04.985270 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:04.985329 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:04.985483 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:04.985583 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:04.985648 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:04.985695 (+ 47us) server_negotiation.cc:299] Negotiation successful
0807 00:44:04.985714 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:44:04.988190 29574 raft_consensus.cc:1261] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 1 FOLLOWER]: Refusing update from remote peer 7175925677f148b0ae4153a5ea1d5fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:04.988190 29377 raft_consensus.cc:1261] T 7ff6d2799c9842dda69f3f1907304ff6 P 7857a79570b1499f98595c814d9f76c5 [term 1 FOLLOWER]: Refusing update from remote peer 7175925677f148b0ae4153a5ea1d5fdf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:04.988564 29885 consensus_queue.cc:1035] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f74746f0f4e4966a31ea61f6ba69c77" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 35551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:04.988808 29891 consensus_queue.cc:1035] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [LEADER]: Connected to new peer: Peer: permanent_uuid: "7857a79570b1499f98595c814d9f76c5" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:04.993428 29145 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:04.993553 29145 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35684:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:04.994547 29145 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f9d9dc551b04fc48048ec6c946dad74: Sending DeleteTablet for 3 replicas of tablet 7ff6d2799c9842dda69f3f1907304ff6
I20260807 00:44:04.995034 29341 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff6d2799c9842dda69f3f1907304ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:04 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:59330
I20260807 00:44:04.995050 29550 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff6d2799c9842dda69f3f1907304ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:04 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:36986
I20260807 00:44:04.995102 29752 tablet_service.cc:1514] Processing DeleteTablet for tablet 7ff6d2799c9842dda69f3f1907304ff6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:04 UTC) from {username='kudu', principal='kudu/127.1.116.190@KRBTEST.COM'} at 127.0.0.1:44156
I20260807 00:44:04.995529 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 29243
I20260807 00:44:05.006917 29919 tablet_replica.cc:330] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: stopping tablet replica
I20260807 00:44:05.007114 29919 raft_consensus.cc:2229] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:44:05.007320 29919 raft_consensus.cc:2258] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:05.012977 29913 tablet_replica.cc:330] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77: stopping tablet replica
W20260807 00:44:05.012965 29126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:41153 (error 108)
I20260807 00:44:05.013103 29913 raft_consensus.cc:2229] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:44:05.013218 29913 raft_consensus.cc:2258] T 7ff6d2799c9842dda69f3f1907304ff6 P 7f74746f0f4e4966a31ea61f6ba69c77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:05.013942 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.013770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:41153 (local address 127.0.0.1:59338)
0807 00:44:05.013831 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.013886 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:41153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 00:44:05.013973 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 29458
W20260807 00:44:05.014089 29126 catalog_manager.cc:4584] TS 7857a79570b1499f98595c814d9f76c5 (127.1.116.129:41153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7ff6d2799c9842dda69f3f1907304ff6: Network error: Client connection negotiation failed: client connection to 127.1.116.129:41153: connect: Connection refused (error 111)
I20260807 00:44:05.023381 29919 ts_tablet_manager.cc:1905] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:44:05.025650 29919 ts_tablet_manager.cc:1918] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:44:05.025774 29919 log.cc:1196] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/7ff6d2799c9842dda69f3f1907304ff6
I20260807 00:44:05.026228 29919 ts_tablet_manager.cc:1939] T 7ff6d2799c9842dda69f3f1907304ff6 P 7175925677f148b0ae4153a5ea1d5fdf: Deleting consensus metadata
I20260807 00:44:05.027072 29124 catalog_manager.cc:4841] TS 7175925677f148b0ae4153a5ea1d5fdf (127.1.116.131:41491): tablet 7ff6d2799c9842dda69f3f1907304ff6 (table test-table [id=4165a2ecba0348cca225326803da82ba]) successfully deleted
I20260807 00:44:05.033823 29870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.033702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:35551 (local address 127.0.0.1:36990)
0807 00:44:05.033760 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.033794 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.130:35551: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260807 00:44:05.034006 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 29650
I20260807 00:44:05.043553 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 29101
2026-08-07T00:44:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2541 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T00:44:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:05Z Disabled control of system clock
I20260807 00:44:05.081468 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:42811
--webserver_interface=127.1.116.190
--webserver_port=0
--builtin_ntp_servers=127.1.116.148:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:42811
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:44:05.173360 29924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:05.173604 29924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:05.173633 29924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:05.174795 29924 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:05.174839 29924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:05.174851 29924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:05.174863 29924 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:05.174876 29924 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:05.176196 29924 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:42811
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:42811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:05.176416 29924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:05.176726 29924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:05.181960 29934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:05.181957 29933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:05.182065 29936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:05.182292 29924 server_base.cc:1034] running on GCE node
I20260807 00:44:05.182687 29924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:05.183032 29924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:05.184242 29924 hybrid_clock.cc:648] HybridClock initialized: now 1786063445184227 us; error 49 us; skew 500 ppm
I20260807 00:44:05.185731 29924 webserver.cc:460] Webserver started at http://127.1.116.190:42305/ using document root <none> and password file <none>
I20260807 00:44:05.186004 29924 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:05.186075 29924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:05.186229 29924 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:05.187554 29924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "6b2fa3871a614f07a9912cf1529760cf"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "1cfb40c506728c6df5ecb2eb8ba15faf"
server_key_iv: "87be9b56ea7c9aac79dab868114f6a6f"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.187979 29924 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "6b2fa3871a614f07a9912cf1529760cf"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "1cfb40c506728c6df5ecb2eb8ba15faf"
server_key_iv: "87be9b56ea7c9aac79dab868114f6a6f"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.189489 29924 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:44:05.190379 29945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.190655 29924 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:05.190778 29924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "6b2fa3871a614f07a9912cf1529760cf"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "1cfb40c506728c6df5ecb2eb8ba15faf"
server_key_iv: "87be9b56ea7c9aac79dab868114f6a6f"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.190867 29924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:05.203935 29924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:05.204290 29924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:05.204452 29924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:05.209662 29924 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:42811
I20260807 00:44:05.209693 30027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:42811 every 8 connection(s)
I20260807 00:44:05.210104 29924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:05.211292 30028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:05.213692 30028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf: Bootstrap starting.
I20260807 00:44:05.214296 30028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:05.214684 30028 log.cc:824] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:05.215449 30028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf: No bootstrap required, opened a new log
I20260807 00:44:05.217058 30028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2fa3871a614f07a9912cf1529760cf" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42811 } }
I20260807 00:44:05.217198 30028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:05.217221 30028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b2fa3871a614f07a9912cf1529760cf, State: Initialized, Role: FOLLOWER
I20260807 00:44:05.217375 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 29924
I20260807 00:44:05.217375 30028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2fa3871a614f07a9912cf1529760cf" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42811 } }
I20260807 00:44:05.217468 30028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:05.217507 30028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:05.217496 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:05.217561 30028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:05.217767 1490 external_mini_cluster.cc:1428] Setting key 36d16aef2c58a647dfc698c1a18b7585
I20260807 00:44:05.218287 30028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2fa3871a614f07a9912cf1529760cf" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42811 } }
I20260807 00:44:05.218403 30028 leader_election.cc:302] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6b2fa3871a614f07a9912cf1529760cf; no voters:
I20260807 00:44:05.218623 30028 leader_election.cc:288] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:05.218704 30034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:05.218899 30028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:05.218930 30034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [term 1 LEADER]: Becoming Leader. State: Replica: 6b2fa3871a614f07a9912cf1529760cf, State: Running, Role: LEADER
I20260807 00:44:05.219038 30034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2fa3871a614f07a9912cf1529760cf" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42811 } }
I20260807 00:44:05.219485 30034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6b2fa3871a614f07a9912cf1529760cf. Latest consensus state: current_term: 1 leader_uuid: "6b2fa3871a614f07a9912cf1529760cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2fa3871a614f07a9912cf1529760cf" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42811 } } }
I20260807 00:44:05.219583 30034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:05.219753 30035 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6b2fa3871a614f07a9912cf1529760cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6b2fa3871a614f07a9912cf1529760cf" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42811 } } }
I20260807 00:44:05.219944 30035 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:05.219864 30049 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:05.220688 30049 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:05.223294 30049 catalog_manager.cc:1347] Generated new cluster ID: 5de47b572dc746b29918a39cca35a282
I20260807 00:44:05.223366 30049 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:05.224824 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.218354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49580 (local address 127.1.116.190:42811)
0807 00:44:05.218719 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.218730 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.218771 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:44:05.219030 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.219037 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.219124 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.219578 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:05.219690 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.221250 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.222279 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.222970 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.223109 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.223863 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:05.223867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:05.223867 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:05.224075 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:05.224121 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:05.224137 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:05.224470 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.224505 (+ 35us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.224607 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":220,"thread_start_us":137,"threads_started":1}
I20260807 00:44:05.246783 30049 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:05.247537 30049 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:05.262313 30049 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf: Generated new TSK 0
I20260807 00:44:05.262655 30049 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:05.271016 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.129:0
--local_ip_for_outbound_sockets=127.1.116.129
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42811
--builtin_ntp_servers=127.1.116.148:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:44:05.367282 30067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:05.367507 30067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:05.367530 30067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:05.368755 30067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:05.368801 30067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:05.368827 30067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.129
I20260807 00:44:05.370216 30067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.116.129
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30067
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.129
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:05.370441 30067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:05.370813 30067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:05.376840 30074 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:05.376875 30079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:05.377007 30067 server_base.cc:1034] running on GCE node
W20260807 00:44:05.376875 30075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:05.377466 30067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:05.377759 30067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:05.378948 30067 hybrid_clock.cc:648] HybridClock initialized: now 1786063445378925 us; error 47 us; skew 500 ppm
I20260807 00:44:05.380379 30067 webserver.cc:460] Webserver started at http://127.1.116.129:41481/ using document root <none> and password file <none>
I20260807 00:44:05.380682 30067 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:05.380757 30067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:05.380926 30067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:05.382095 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data/instance:
uuid: "f637326c3b9648f898662d6cf3bc1b4e"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "c0e09e11ce73197310eb4c504cd4e4bb"
server_key_iv: "2cfbd18176d490a964cf7ad053da5ccc"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.382463 30067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance:
uuid: "f637326c3b9648f898662d6cf3bc1b4e"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "c0e09e11ce73197310eb4c504cd4e4bb"
server_key_iv: "2cfbd18176d490a964cf7ad053da5ccc"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.383919 30067 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:05.384738 30087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.385018 30067 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:05.385150 30067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/wal
uuid: "f637326c3b9648f898662d6cf3bc1b4e"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "c0e09e11ce73197310eb4c504cd4e4bb"
server_key_iv: "2cfbd18176d490a964cf7ad053da5ccc"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.385241 30067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:05.393549 30067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:05.393913 30067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:05.394064 30067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:05.394296 30067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:05.394704 30067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:05.394761 30067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.394795 30067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:05.394822 30067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.401649 30067 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.129:44437
I20260807 00:44:05.401759 30253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.129:44437 every 8 connection(s)
I20260807 00:44:05.402074 30067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/data/info.pb
I20260807 00:44:05.408453 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.402981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.129:36395 (local address 127.1.116.190:42811)
0807 00:44:05.403084 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.403087 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.403330 (+ 243us) server_negotiation.cc:407] Connection header received
0807 00:44:05.403965 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.403967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.403986 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.404060 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:05.404898 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.405414 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.406706 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.407041 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.407112 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.408061 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:05.408063 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:05.408063 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:05.408069 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:05.408111 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:05.408116 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:05.408181 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.408354 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.408390 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 00:44:05.408578 30261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.402916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42811 (local address 127.1.116.129:36395)
0807 00:44:05.403167 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.403208 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.403814 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.404122 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.404125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.404135 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:05.404750 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.404755 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.405536 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.405538 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.406463 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.406466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.407140 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.407141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.407302 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.407881 (+ 579us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:44:05.407882 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:05.407914 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:44:05.407940 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:05.408205 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:05.408211 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:05.408213 (+ 2us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.408272 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.408370 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":115,"threads_started":1}
I20260807 00:44:05.409044 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 30067
I20260807 00:44:05.409142 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-0/wal/instance
I20260807 00:44:05.409314 1490 external_mini_cluster.cc:1428] Setting key eacab43be45933593ac1667a66fece91
I20260807 00:44:05.410136 30257 heartbeater.cc:347] Connected to a master server at 127.1.116.190:42811
I20260807 00:44:05.410259 30257 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:05.410499 30257 heartbeater.cc:511] Master 127.1.116.190:42811 requested a full tablet report, sending...
I20260807 00:44:05.410748 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.130:0
--local_ip_for_outbound_sockets=127.1.116.130
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42811
--builtin_ntp_servers=127.1.116.148:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:44:05.411154 29974 ts_manager.cc:194] Registered new tserver with Master: f637326c3b9648f898662d6cf3bc1b4e (127.1.116.129:44437)
I20260807 00:44:05.412022 29974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.129:36395
W20260807 00:44:05.498467 30262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:05.498723 30262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:05.498745 30262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:05.500013 30262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:05.500069 30262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:05.500108 30262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.130
I20260807 00:44:05.501696 30262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.116.130
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.130
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:05.501935 30262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:05.502241 30262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:05.507259 30276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:05.507308 30279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:05.507440 30277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:05.507750 30262 server_base.cc:1034] running on GCE node
I20260807 00:44:05.507992 30262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:05.508292 30262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:05.509480 30262 hybrid_clock.cc:648] HybridClock initialized: now 1786063445509464 us; error 43 us; skew 500 ppm
I20260807 00:44:05.511026 30262 webserver.cc:460] Webserver started at http://127.1.116.130:40635/ using document root <none> and password file <none>
I20260807 00:44:05.511327 30262 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:05.511404 30262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:05.511574 30262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:05.512755 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data/instance:
uuid: "3b29d145ae16470d9d9db4bbab5d9637"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "5b7568e6529cfb3278dd074bd3d74415"
server_key_iv: "b4e9ed39baac31f89d7df0ca58542fdc"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.513221 30262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance:
uuid: "3b29d145ae16470d9d9db4bbab5d9637"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "5b7568e6529cfb3278dd074bd3d74415"
server_key_iv: "b4e9ed39baac31f89d7df0ca58542fdc"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.515115 30262 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260807 00:44:05.515981 30287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.516327 30262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:05.516434 30262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal
uuid: "3b29d145ae16470d9d9db4bbab5d9637"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "5b7568e6529cfb3278dd074bd3d74415"
server_key_iv: "b4e9ed39baac31f89d7df0ca58542fdc"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.516495 30262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:05.537498 30262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:05.537815 30262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:05.537933 30262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:05.538165 30262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:05.538623 30262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:05.538689 30262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.538729 30262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:05.538759 30262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.546077 30262 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.130:32801
I20260807 00:44:05.546136 30460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.130:32801 every 8 connection(s)
I20260807 00:44:05.546566 30262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/data/info.pb
I20260807 00:44:05.547262 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 30262
I20260807 00:44:05.547400 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal/instance
I20260807 00:44:05.547672 1490 external_mini_cluster.cc:1428] Setting key 715f42cc78b6d11852f72d61f9fd6e3f
I20260807 00:44:05.548954 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.116.131:0
--local_ip_for_outbound_sockets=127.1.116.131
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42811
--builtin_ntp_servers=127.1.116.148:39603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:44:05.553718 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.547475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.130:58803 (local address 127.1.116.190:42811)
0807 00:44:05.547536 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.547538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.547828 (+ 290us) server_negotiation.cc:407] Connection header received
0807 00:44:05.548682 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.548683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.548693 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.548735 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:05.549865 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.550404 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.551885 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.552240 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.552337 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.553386 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:05.553387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:05.553388 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:05.553392 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:05.553407 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:05.553411 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:05.553462 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.553623 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.553654 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:44:05.554010 30465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.547361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42811 (local address 127.1.116.130:58803)
0807 00:44:05.547633 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.547685 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.548534 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.548833 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.548838 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.548849 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:05.549687 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.549695 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.550518 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.550519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.551746 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.551749 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.552377 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.552378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.552511 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.553200 (+ 689us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:44:05.553201 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:05.553246 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:44:05.553278 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:05.553489 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:05.553495 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:05.553499 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.553577 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.553707 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":133,"threads_started":1}
I20260807 00:44:05.554643 30462 heartbeater.cc:347] Connected to a master server at 127.1.116.190:42811
I20260807 00:44:05.554752 30462 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:05.554984 30462 heartbeater.cc:511] Master 127.1.116.190:42811 requested a full tablet report, sending...
I20260807 00:44:05.555470 29974 ts_manager.cc:194] Registered new tserver with Master: 3b29d145ae16470d9d9db4bbab5d9637 (127.1.116.130:32801)
I20260807 00:44:05.556078 29974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.130:58803
W20260807 00:44:05.643556 30466 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:05.643793 30466 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:05.643816 30466 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:05.644942 30466 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:05.644981 30466 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:05.645006 30466 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.116.131
I20260807 00:44:05.646451 30466 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:39603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.116.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.116.131
--webserver_port=0
--tserver_master_addrs=127.1.116.190:42811
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30466
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.116.131
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:05.646734 30466 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:05.647027 30466 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:05.652653 30475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:05.652670 30472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:05.652670 30478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:05.653196 30466 server_base.cc:1034] running on GCE node
I20260807 00:44:05.653443 30466 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:05.653718 30466 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:05.654892 30466 hybrid_clock.cc:648] HybridClock initialized: now 1786063445654891 us; error 53 us; skew 500 ppm
I20260807 00:44:05.656328 30466 webserver.cc:460] Webserver started at http://127.1.116.131:38739/ using document root <none> and password file <none>
I20260807 00:44:05.656679 30466 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:05.656751 30466 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:05.656919 30466 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:05.658114 30466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data/instance:
uuid: "595360733a7643a29656a2601d55ff08"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "53aa68e781c01ad07bac467fdd6f6a50"
server_key_iv: "bf858e6a8a1b71bc2f73c86317063d06"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.658514 30466 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance:
uuid: "595360733a7643a29656a2601d55ff08"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "53aa68e781c01ad07bac467fdd6f6a50"
server_key_iv: "bf858e6a8a1b71bc2f73c86317063d06"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.660043 30466 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:05.660948 30484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.661234 30466 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:05.661355 30466 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal
uuid: "595360733a7643a29656a2601d55ff08"
format_stamp: "Formatted at 2026-08-07 00:44:05 on dist-test-slave-h6n0"
server_key: "53aa68e781c01ad07bac467fdd6f6a50"
server_key_iv: "bf858e6a8a1b71bc2f73c86317063d06"
server_key_version: "encryptionkey@0"
I20260807 00:44:05.661440 30466 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:05.673781 30466 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:05.674129 30466 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:05.674297 30466 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:05.674606 30466 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:44:05.675004 30466 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:44:05.675060 30466 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.675104 30466 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:44:05.675132 30466 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:05.682874 30466 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.131:35445
I20260807 00:44:05.682971 30661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.131:35445 every 8 connection(s)
I20260807 00:44:05.683401 30466 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/data/info.pb
I20260807 00:44:05.687162 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 30466
I20260807 00:44:05.687290 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal/instance
I20260807 00:44:05.687553 1490 external_mini_cluster.cc:1428] Setting key 798042cdabea30fa51866c55f745407a
I20260807 00:44:05.690587 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.685103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:56407 (local address 127.1.116.190:42811)
0807 00:44:05.685203 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.685206 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.685508 (+ 302us) server_negotiation.cc:407] Connection header received
0807 00:44:05.686092 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.686093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.686105 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.686169 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:05.687071 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.687859 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.689113 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.689323 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.689409 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.690267 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:05.690268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:05.690269 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:05.690274 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:05.690288 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:05.690292 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:05.690341 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.690446 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.690476 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:44:05.690860 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.685097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42811 (local address 127.1.116.131:56407)
0807 00:44:05.685351 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.685392 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.685989 (+ 597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.686236 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.686240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.686249 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:05.686965 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.686972 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.687948 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.687950 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.689023 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.689026 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.689422 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.689423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.689558 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.690099 (+ 541us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:44:05.690100 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:05.690129 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:44:05.690152 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:05.690353 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:05.690358 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:05.690363 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.690458 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.690584 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":140,"threads_started":1}
I20260807 00:44:05.691371 30662 heartbeater.cc:347] Connected to a master server at 127.1.116.190:42811
I20260807 00:44:05.691478 30662 heartbeater.cc:464] Registering TS with master...
I20260807 00:44:05.691648 30662 heartbeater.cc:511] Master 127.1.116.190:42811 requested a full tablet report, sending...
I20260807 00:44:05.692070 29974 ts_manager.cc:194] Registered new tserver with Master: 595360733a7643a29656a2601d55ff08 (127.1.116.131:35445)
I20260807 00:44:05.692564 29974 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.116.131:56407
I20260807 00:44:05.699822 1490 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:44:05.704581 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.701394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49588 (local address 127.1.116.190:42811)
0807 00:44:05.701476 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.701478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.701572 (+ 94us) server_negotiation.cc:407] Connection header received
0807 00:44:05.701692 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.701694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.701710 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.701763 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:05.701968 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.702482 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.703331 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.703535 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.703611 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.704320 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:05.704322 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:05.704322 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:05.704326 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:05.704341 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:05.704345 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:05.704400 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.704462 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.704487 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:44:05.706439 29974 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49588:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:44:05.708340 29974 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:44:05.715415 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.710898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.131:35445 (local address 127.0.0.1:58532)
0807 00:44:05.711057 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.711101 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.711175 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.711709 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.711712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.711716 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:05.711798 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.711800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.713200 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.713200 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.714326 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.714328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.715112 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.715113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.715192 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.715206 (+ 14us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.715276 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.715337 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":100,"threads_started":1}
I20260807 00:44:05.715574 30679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.711173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:32801 (local address 127.0.0.1:41772)
0807 00:44:05.711471 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.711482 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.711548 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.711842 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.711843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.711846 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:05.711913 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.711915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.713346 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.713347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.714353 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.714356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.715296 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.715298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.715362 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.715368 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.715403 (+ 35us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.715459 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260807 00:44:05.715576 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.710948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44437 (local address 127.0.0.1:39954)
0807 00:44:05.711131 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.711145 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.711214 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.712011 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.712012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.712015 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:05.712061 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.712063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.712894 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.712895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.714181 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.714184 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.715313 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.715314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.715373 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.715377 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.715407 (+ 30us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.715459 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":2,"thread_start_us":73,"threads_started":1}
I20260807 00:44:05.715766 30678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.711093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58532 (local address 127.1.116.131:35445)
0807 00:44:05.711516 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.711521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.711533 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:44:05.711583 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.711585 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.711602 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.711695 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:05.711908 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.713094 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.714423 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.715013 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.715108 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.715195 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.715627 (+ 432us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.715671 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":106,"threads_started":1}
I20260807 00:44:05.716106 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.711280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39954 (local address 127.1.116.129:44437)
0807 00:44:05.711830 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.711834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.711844 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:44:05.711883 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.711885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.711901 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.711989 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:05.712134 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.712790 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.714308 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.715192 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.715620 (+ 428us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.715742 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.715960 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.716017 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":122,"threads_started":1}
I20260807 00:44:05.716135 30684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.711442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.1.116.130:32801)
0807 00:44:05.711662 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.711667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.711677 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:44:05.711722 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.711724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.711741 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.711833 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:05.712180 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.713259 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.714460 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.715178 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.715886 (+ 708us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.715982 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.716006 (+ 24us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.716043 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":117,"threads_started":1}
I20260807 00:44:05.716548 30572 tablet_service.cc:1467] Processing CreateTablet for tablet 381682baf7074e33aebb8d11aba29489 (DEFAULT_TABLE table=test-table [id=d8afb16a82bb488ea9a3f6f0312a9776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:05.716826 30363 tablet_service.cc:1467] Processing CreateTablet for tablet 381682baf7074e33aebb8d11aba29489 (DEFAULT_TABLE table=test-table [id=d8afb16a82bb488ea9a3f6f0312a9776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:05.716956 30572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381682baf7074e33aebb8d11aba29489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:05.717185 30363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381682baf7074e33aebb8d11aba29489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:05.719359 30160 tablet_service.cc:1467] Processing CreateTablet for tablet 381682baf7074e33aebb8d11aba29489 (DEFAULT_TABLE table=test-table [id=d8afb16a82bb488ea9a3f6f0312a9776]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:44:05.719866 30160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 381682baf7074e33aebb8d11aba29489. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:05.720912 30688 tablet_bootstrap.cc:492] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Bootstrap starting.
I20260807 00:44:05.720922 30686 tablet_bootstrap.cc:492] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Bootstrap starting.
I20260807 00:44:05.721511 30688 tablet_bootstrap.cc:654] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:05.721735 30686 tablet_bootstrap.cc:654] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:05.721864 30688 log.cc:824] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:05.722132 30686 log.cc:824] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:05.722738 30688 tablet_bootstrap.cc:492] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: No bootstrap required, opened a new log
I20260807 00:44:05.722819 30688 ts_tablet_manager.cc:1397] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:44:05.723004 30686 tablet_bootstrap.cc:492] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: No bootstrap required, opened a new log
I20260807 00:44:05.723091 30686 ts_tablet_manager.cc:1397] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:44:05.724159 30696 tablet_bootstrap.cc:492] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e: Bootstrap starting.
I20260807 00:44:05.724745 30696 tablet_bootstrap.cc:654] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:05.725063 30696 log.cc:824] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:05.725621 30688 raft_consensus.cc:348] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.725621 30686 raft_consensus.cc:348] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.725782 30696 tablet_bootstrap.cc:492] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e: No bootstrap required, opened a new log
I20260807 00:44:05.725800 30686 raft_consensus.cc:374] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:05.725800 30688 raft_consensus.cc:374] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:05.725841 30686 raft_consensus.cc:729] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 595360733a7643a29656a2601d55ff08, State: Initialized, Role: FOLLOWER
I20260807 00:44:05.725841 30688 raft_consensus.cc:729] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b29d145ae16470d9d9db4bbab5d9637, State: Initialized, Role: FOLLOWER
I20260807 00:44:05.725870 30696 ts_tablet_manager.cc:1397] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:44:05.725991 30688 consensus_queue.cc:260] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.725991 30686 consensus_queue.cc:260] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.726253 30686 ts_tablet_manager.cc:1428] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:05.726253 30688 ts_tablet_manager.cc:1428] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:05.726372 30462 heartbeater.cc:503] Master 127.1.116.190:42811 was elected leader, sending a full tablet report...
I20260807 00:44:05.726372 30662 heartbeater.cc:503] Master 127.1.116.190:42811 was elected leader, sending a full tablet report...
I20260807 00:44:05.728528 30696 raft_consensus.cc:348] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.728703 30696 raft_consensus.cc:374] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:05.728754 30696 raft_consensus.cc:729] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f637326c3b9648f898662d6cf3bc1b4e, State: Initialized, Role: FOLLOWER
I20260807 00:44:05.728883 30696 consensus_queue.cc:260] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.729159 30696 ts_tablet_manager.cc:1428] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260807 00:44:05.729238 30257 heartbeater.cc:503] Master 127.1.116.190:42811 was elected leader, sending a full tablet report...
W20260807 00:44:05.797169 30463 tablet.cc:2369] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:44:05.805555 30701 raft_consensus.cc:482] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:44:05.805678 30701 raft_consensus.cc:504] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.806119 30701 leader_election.cc:288] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f637326c3b9648f898662d6cf3bc1b4e (127.1.116.129:44437), 3b29d145ae16470d9d9db4bbab5d9637 (127.1.116.130:32801)
I20260807 00:44:05.809523 30681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.806462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:37541 (local address 127.1.116.129:44437)
0807 00:44:05.806518 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.806520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.806556 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:44:05.806673 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.806674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.806690 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.806743 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:05.807055 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.807593 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.808751 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.809121 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.809185 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.809219 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.809387 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.809439 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260807 00:44:05.809517 30664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.806319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44437 (local address 127.1.116.131:37541)
0807 00:44:05.806402 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.806422 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.806550 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.806819 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.806820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.806822 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:05.806931 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.806934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.807691 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.807692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.808637 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.808640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.809220 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.809221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.809302 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.809305 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.809351 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.809393 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":52}
I20260807 00:44:05.809589 30684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.806725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.116.131:51641 (local address 127.1.116.130:32801)
0807 00:44:05.806774 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.806777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.806786 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:44:05.806818 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.806819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.806833 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.806901 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:44:05.807142 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.807677 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.808746 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.809135 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.809233 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.809244 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.809447 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.809498 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:44:05.809580 30704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.806431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.130:32801 (local address 127.1.116.131:51641)
0807 00:44:05.806681 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.806693 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:05.806750 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:05.806916 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:05.806917 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:05.806919 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:44:05.807045 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.807047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.807766 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.807767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.808635 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:05.808637 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.809274 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.809275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:05.809329 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.809354 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:44:05.809421 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:44:05.809466 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":2,"thread_start_us":153,"threads_started":1}
I20260807 00:44:05.810039 30188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381682baf7074e33aebb8d11aba29489" candidate_uuid: "595360733a7643a29656a2601d55ff08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f637326c3b9648f898662d6cf3bc1b4e" is_pre_election: true
I20260807 00:44:05.810042 30395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381682baf7074e33aebb8d11aba29489" candidate_uuid: "595360733a7643a29656a2601d55ff08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b29d145ae16470d9d9db4bbab5d9637" is_pre_election: true
I20260807 00:44:05.810204 30188 raft_consensus.cc:2454] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595360733a7643a29656a2601d55ff08 in term 0.
I20260807 00:44:05.810204 30395 raft_consensus.cc:2454] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 595360733a7643a29656a2601d55ff08 in term 0.
I20260807 00:44:05.810428 30493 leader_election.cc:302] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595360733a7643a29656a2601d55ff08, f637326c3b9648f898662d6cf3bc1b4e; no voters:
I20260807 00:44:05.810585 30701 raft_consensus.cc:2781] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:44:05.810633 30701 raft_consensus.cc:482] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:44:05.810662 30701 raft_consensus.cc:3037] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:05.811368 30701 raft_consensus.cc:504] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.811499 30701 leader_election.cc:288] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [CANDIDATE]: Term 1 election: Requested vote from peers f637326c3b9648f898662d6cf3bc1b4e (127.1.116.129:44437), 3b29d145ae16470d9d9db4bbab5d9637 (127.1.116.130:32801)
I20260807 00:44:05.811712 30188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381682baf7074e33aebb8d11aba29489" candidate_uuid: "595360733a7643a29656a2601d55ff08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f637326c3b9648f898662d6cf3bc1b4e"
I20260807 00:44:05.811791 30188 raft_consensus.cc:3037] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:05.811760 30395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "381682baf7074e33aebb8d11aba29489" candidate_uuid: "595360733a7643a29656a2601d55ff08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b29d145ae16470d9d9db4bbab5d9637"
I20260807 00:44:05.811908 30395 raft_consensus.cc:3037] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:05.812623 30188 raft_consensus.cc:2454] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595360733a7643a29656a2601d55ff08 in term 1.
I20260807 00:44:05.812805 30493 leader_election.cc:302] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 595360733a7643a29656a2601d55ff08, f637326c3b9648f898662d6cf3bc1b4e; no voters:
I20260807 00:44:05.812920 30701 raft_consensus.cc:2781] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:05.813009 30395 raft_consensus.cc:2454] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 595360733a7643a29656a2601d55ff08 in term 1.
I20260807 00:44:05.813189 30701 raft_consensus.cc:686] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 1 LEADER]: Becoming Leader. State: Replica: 595360733a7643a29656a2601d55ff08, State: Running, Role: LEADER
I20260807 00:44:05.813318 30701 consensus_queue.cc:237] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } }
I20260807 00:44:05.814297 29974 catalog_manager.cc:5495] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 595360733a7643a29656a2601d55ff08 (127.1.116.131). New cstate: current_term: 1 leader_uuid: "595360733a7643a29656a2601d55ff08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "595360733a7643a29656a2601d55ff08" member_type: VOTER last_known_addr { host: "127.1.116.131" port: 35445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 } health_report { overall_health: UNKNOWN } } }
I20260807 00:44:05.834498 30678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.832073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58540 (local address 127.1.116.131:35445)
0807 00:44:05.832130 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:05.832132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:05.832144 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:44:05.832193 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:05.832194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:05.832211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:05.832281 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:05.832436 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.832994 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.833816 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:05.833993 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:05.834054 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:05.834194 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:05.834285 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:05.834347 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:05.834416 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 00:44:05.834431 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 00:44:05.836683 30188 raft_consensus.cc:1261] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 1 FOLLOWER]: Refusing update from remote peer 595360733a7643a29656a2601d55ff08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:05.836782 30395 raft_consensus.cc:1261] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 1 FOLLOWER]: Refusing update from remote peer 595360733a7643a29656a2601d55ff08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:05.837061 30701 consensus_queue.cc:1035] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f637326c3b9648f898662d6cf3bc1b4e" member_type: VOTER last_known_addr { host: "127.1.116.129" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:05.837291 30706 consensus_queue.cc:1035] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b29d145ae16470d9d9db4bbab5d9637" member_type: VOTER last_known_addr { host: "127.1.116.130" port: 32801 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:05.839881 30712 mvcc.cc:205] Tried to move back new op lower bound from 7315715874144333824 to 7315715874052276224. Current Snapshot: MvccSnapshot[applied={T|T < 7315715874144333824}]
I20260807 00:44:05.840481 30713 mvcc.cc:205] Tried to move back new op lower bound from 7315715874144333824 to 7315715874052276224. Current Snapshot: MvccSnapshot[applied={T|T < 7315715874144333824}]
I20260807 00:44:05.841223 30714 mvcc.cc:205] Tried to move back new op lower bound from 7315715874144333824 to 7315715874052276224. Current Snapshot: MvccSnapshot[applied={T|T < 7315715874144333824}]
I20260807 00:44:05.842476 29974 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:05.842615 29974 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49588:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:44:05.843493 29974 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6b2fa3871a614f07a9912cf1529760cf: Sending DeleteTablet for 3 replicas of tablet 381682baf7074e33aebb8d11aba29489
I20260807 00:44:05.843791 30572 tablet_service.cc:1514] Processing DeleteTablet for tablet 381682baf7074e33aebb8d11aba29489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:05 UTC) from {username='slave'} at 127.0.0.1:58532
I20260807 00:44:05.843990 30160 tablet_service.cc:1514] Processing DeleteTablet for tablet 381682baf7074e33aebb8d11aba29489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:05 UTC) from {username='slave'} at 127.0.0.1:39954
I20260807 00:44:05.844273 30728 tablet_replica.cc:330] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e: stopping tablet replica
I20260807 00:44:05.844259 30363 tablet_service.cc:1514] Processing DeleteTablet for tablet 381682baf7074e33aebb8d11aba29489 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:44:05 UTC) from {username='slave'} at 127.0.0.1:41772
I20260807 00:44:05.844375 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 30067
I20260807 00:44:05.844498 30728 raft_consensus.cc:2229] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:44:05.844656 30728 raft_consensus.cc:2258] T 381682baf7074e33aebb8d11aba29489 P f637326c3b9648f898662d6cf3bc1b4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:05.849396 30729 tablet_replica.cc:330] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: stopping tablet replica
I20260807 00:44:05.849531 30729 raft_consensus.cc:2229] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:44:05.849673 30729 raft_consensus.cc:2258] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:05.851073 30729 ts_tablet_manager.cc:1905] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:44:05.853080 30729 ts_tablet_manager.cc:1918] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:44:05.853175 30729 log.cc:1196] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-1/wal/wals/381682baf7074e33aebb8d11aba29489
I20260807 00:44:05.853300 30727 tablet_replica.cc:330] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: stopping tablet replica
I20260807 00:44:05.853458 30727 raft_consensus.cc:2229] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:44:05.853480 30729 ts_tablet_manager.cc:1939] T 381682baf7074e33aebb8d11aba29489 P 3b29d145ae16470d9d9db4bbab5d9637: Deleting consensus metadata
I20260807 00:44:05.853632 30727 raft_consensus.cc:2258] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:44:05.853953 29952 catalog_manager.cc:4841] TS 3b29d145ae16470d9d9db4bbab5d9637 (127.1.116.130:32801): tablet 381682baf7074e33aebb8d11aba29489 (table test-table [id=d8afb16a82bb488ea9a3f6f0312a9776]) successfully deleted
I20260807 00:44:05.854238 30727 ts_tablet_manager.cc:1905] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 00:44:05.855947 29952 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.116.129:44437 (error 108)
I20260807 00:44:05.856289 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 30262
I20260807 00:44:05.875120 30677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:05.872034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.129:44437 (local address 127.0.0.1:39966)
0807 00:44:05.874983 (+ 2949us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:05.875069 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.129:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2911}
W20260807 00:44:05.875706 29952 catalog_manager.cc:4584] TS f637326c3b9648f898662d6cf3bc1b4e (127.1.116.129:44437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 381682baf7074e33aebb8d11aba29489: Network error: Client connection negotiation failed: client connection to 127.1.116.129:44437: connect: Connection refused (error 111)
I20260807 00:44:05.884112 30727 ts_tablet_manager.cc:1918] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:44:05.884341 30727 log.cc:1196] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Deleting WAL directory at /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786063190385254-1490-0/minicluster-data/ts-2/wal/wals/381682baf7074e33aebb8d11aba29489
I20260807 00:44:05.884799 30727 ts_tablet_manager.cc:1939] T 381682baf7074e33aebb8d11aba29489 P 595360733a7643a29656a2601d55ff08: Deleting consensus metadata
I20260807 00:44:05.885526 29949 catalog_manager.cc:4841] TS 595360733a7643a29656a2601d55ff08 (127.1.116.131:35445): tablet 381682baf7074e33aebb8d11aba29489 (table test-table [id=d8afb16a82bb488ea9a3f6f0312a9776]) successfully deleted
I20260807 00:44:05.889838 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 30466
I20260807 00:44:05.903635 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 29924
2026-08-07T00:44:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (857 ms)
[----------] 6 tests from EncryptionPolicyTest (9612 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:05 dist-test-slave-h6n0 krb5kdc[30738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:05 dist-test-slave-h6n0 krb5kdc[30738](info): set up 2 sockets
Aug 07 00:44:05 dist-test-slave-h6n0 krb5kdc[30738](info): commencing operation
krb5kdc: starting...
W20260807 00:44:07.162566 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.234s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:07 dist-test-slave-h6n0 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:07.297890 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39979
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36409
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39979
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:07.388788 30757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:07.389070 30757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:07.389107 30757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:07.390259 30757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:07.390340 30757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:07.390369 30757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:07.390401 30757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:07.390425 30757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:07.391981 30757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36409
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39979
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:07.392293 30757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:07.392575 30757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:07.398015 30766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:07.398203 30757 server_base.cc:1034] running on GCE node
W20260807 00:44:07.398070 30770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:07.398015 30768 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:07.398736 30757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:07.399058 30757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:07.400247 30757 hybrid_clock.cc:648] HybridClock initialized: now 1786063447400228 us; error 58 us; skew 500 ppm
Aug 07 00:44:07 dist-test-slave-h6n0 krb5kdc[30738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063447, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:07.407743 30757 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:07.408186 30757 webserver.cc:460] Webserver started at http://127.0.0.1:34105/ using document root <none> and password file <none>
I20260807 00:44:07.408397 30757 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:07.408442 30757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:07.408535 30757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:07.409590 30757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "7629e9921e804cf481f3fc132dd558cb"
format_stamp: "Formatted at 2026-08-07 00:44:07 on dist-test-slave-h6n0"
server_key: "4cce5ddf3cddc9fe6a03c3f0d36b9176"
server_key_iv: "ec2a5e9a3799758a1f4a656e859c93cb"
server_key_version: "encryptionkey@0"
I20260807 00:44:07.409910 30757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "7629e9921e804cf481f3fc132dd558cb"
format_stamp: "Formatted at 2026-08-07 00:44:07 on dist-test-slave-h6n0"
server_key: "4cce5ddf3cddc9fe6a03c3f0d36b9176"
server_key_iv: "ec2a5e9a3799758a1f4a656e859c93cb"
server_key_version: "encryptionkey@0"
I20260807 00:44:07.411468 30757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:44:07.412339 30780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:07.412619 30757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:07.412741 30757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "7629e9921e804cf481f3fc132dd558cb"
format_stamp: "Formatted at 2026-08-07 00:44:07 on dist-test-slave-h6n0"
server_key: "4cce5ddf3cddc9fe6a03c3f0d36b9176"
server_key_iv: "ec2a5e9a3799758a1f4a656e859c93cb"
server_key_version: "encryptionkey@0"
I20260807 00:44:07.412817 30757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:07.448099 30757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:07.450157 30757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:07.450317 30757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:07.454880 30757 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39979
I20260807 00:44:07.454919 30857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39979 every 8 connection(s)
I20260807 00:44:07.455250 30757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:07.456073 30861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:07.458236 30861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb: Bootstrap starting.
I20260807 00:44:07.458876 30861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:07.459118 30861 log.cc:824] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:07.459775 30861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb: No bootstrap required, opened a new log
I20260807 00:44:07.461319 30861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7629e9921e804cf481f3fc132dd558cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39979 } }
I20260807 00:44:07.461465 30861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:07.461489 30861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7629e9921e804cf481f3fc132dd558cb, State: Initialized, Role: FOLLOWER
I20260807 00:44:07.461621 30861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7629e9921e804cf481f3fc132dd558cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39979 } }
I20260807 00:44:07.461709 30861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:07.461732 30861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:07.461758 30861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:07.462481 30861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7629e9921e804cf481f3fc132dd558cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39979 } }
I20260807 00:44:07.462632 30861 leader_election.cc:302] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7629e9921e804cf481f3fc132dd558cb; no voters:
I20260807 00:44:07.462787 30861 leader_election.cc:288] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:07.462813 30865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:07.462965 30865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [term 1 LEADER]: Becoming Leader. State: Replica: 7629e9921e804cf481f3fc132dd558cb, State: Running, Role: LEADER
I20260807 00:44:07.463115 30861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:07.463179 30865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7629e9921e804cf481f3fc132dd558cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39979 } }
I20260807 00:44:07.463938 30866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7629e9921e804cf481f3fc132dd558cb. Latest consensus state: current_term: 1 leader_uuid: "7629e9921e804cf481f3fc132dd558cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7629e9921e804cf481f3fc132dd558cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39979 } } }
I20260807 00:44:07.463965 30867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7629e9921e804cf481f3fc132dd558cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7629e9921e804cf481f3fc132dd558cb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39979 } } }
I20260807 00:44:07.464263 30867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:07.464423 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 30757
I20260807 00:44:07.464566 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:07.464598 30879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:07.464807 1490 external_mini_cluster.cc:1428] Setting key 66e477f516f7e3d44029e9daf941bb5c
I20260807 00:44:07.464803 30866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:07.465195 30879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:07.468386 30879 catalog_manager.cc:1347] Generated new cluster ID: 8a509d20873c466f82e38e13a0877ef6
I20260807 00:44:07.468446 30879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:44:07 dist-test-slave-h6n0 krb5kdc[30738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063447, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:44:07.476785 30895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:07.466305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58145 (local address 127.0.0.1:39979)
0807 00:44:07.466614 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:07.466623 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:07.466658 (+ 35us) server_negotiation.cc:407] Connection header received
0807 00:44:07.466780 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:07.466788 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:07.466853 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:07.467020 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:07.467990 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:07.469153 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:07.469976 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:07.470450 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:07.470604 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:07.473201 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:07.473205 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:07.473206 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:07.473218 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:07.475079 (+ 1861us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:07.475492 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:07.475497 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:07.475497 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:07.475619 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:07.475864 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:07.475865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:07.475865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:07.476139 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:07.476350 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:07.476440 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 00:44:07.476555 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":91,"threads_started":1}
I20260807 00:44:07.477948 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:07.481384 30879 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:07.482050 30879 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:07.487332 30895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:07.479667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44621 (local address 127.0.0.1:39979)
0807 00:44:07.479708 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:07.479709 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:07.479820 (+ 111us) server_negotiation.cc:407] Connection header received
0807 00:44:07.479885 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:07.479886 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:07.479896 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:07.479968 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:07.480647 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:07.481283 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:07.482290 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:07.482692 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:07.482777 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:07.484186 (+ 1409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:07.484187 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:07.484187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:07.484195 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:07.485993 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:07.486409 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:07.486410 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:07.486411 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:07.486474 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:07.486770 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:07.486771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:07.486771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:07.486913 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:07.487050 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:07.487158 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 00:44:07.487227 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:44:07.489048 30879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7629e9921e804cf481f3fc132dd558cb: Generated new TSK 0
I20260807 00:44:07.489256 30879 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:07.494273 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 30757
W20260807 00:44:07.507340 30733 connection.cc:535] client connection to 127.0.0.1:39979 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39979): Connection reset by peer (error 104)
2026-08-07T00:44:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1592 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:07 dist-test-slave-h6n0 krb5kdc[30912](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:07 dist-test-slave-h6n0 krb5kdc[30912](info): set up 2 sockets
Aug 07 00:44:07 dist-test-slave-h6n0 krb5kdc[30912](info): commencing operation
krb5kdc: starting...
W20260807 00:44:08.742239 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:08 dist-test-slave-h6n0 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:08.879740 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37189
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:08.966213 30932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:08.966466 30932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:08.966495 30932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:08.967895 30932 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:08.967945 30932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:08.967962 30932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:08.967976 30932 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:08.967988 30932 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:08.969522 30932 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37189
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:08.969748 30932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:08.969946 30932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:08.975313 30943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:08.975334 30937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:08.975278 30938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:08.975520 30932 server_base.cc:1034] running on GCE node
I20260807 00:44:08.975812 30932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:08.976174 30932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:08.977332 30932 hybrid_clock.cc:648] HybridClock initialized: now 1786063448977307 us; error 47 us; skew 500 ppm
Aug 07 00:44:08 dist-test-slave-h6n0 krb5kdc[30912](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063448, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:08.980100 30932 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:08.980502 30932 webserver.cc:460] Webserver started at http://127.0.0.1:42091/ using document root <none> and password file <none>
I20260807 00:44:08.980756 30932 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:08.980829 30932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:08.980976 30932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:08.982049 30932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "188ee24c17824a93a083238c639c2b8b"
format_stamp: "Formatted at 2026-08-07 00:44:08 on dist-test-slave-h6n0"
server_key: "0d8b1898584d47e4c4fb61cc594c6aa5"
server_key_iv: "0f3ffa2aa23cf31228e19e5766525e8d"
server_key_version: "encryptionkey@0"
I20260807 00:44:08.982427 30932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "188ee24c17824a93a083238c639c2b8b"
format_stamp: "Formatted at 2026-08-07 00:44:08 on dist-test-slave-h6n0"
server_key: "0d8b1898584d47e4c4fb61cc594c6aa5"
server_key_iv: "0f3ffa2aa23cf31228e19e5766525e8d"
server_key_version: "encryptionkey@0"
I20260807 00:44:08.983915 30932 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:08.984792 30951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:08.985015 30932 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:08.985138 30932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "188ee24c17824a93a083238c639c2b8b"
format_stamp: "Formatted at 2026-08-07 00:44:08 on dist-test-slave-h6n0"
server_key: "0d8b1898584d47e4c4fb61cc594c6aa5"
server_key_iv: "0f3ffa2aa23cf31228e19e5766525e8d"
server_key_version: "encryptionkey@0"
I20260807 00:44:08.985222 30932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:09.001369 30932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:09.003681 30932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:09.003839 30932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:09.008548 30932 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37189
I20260807 00:44:09.008632 31038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37189 every 8 connection(s)
I20260807 00:44:09.008975 30932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:09.010175 31039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:09.012946 31039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b: Bootstrap starting.
I20260807 00:44:09.013613 31039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:09.013897 31039 log.cc:824] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:09.014809 31039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b: No bootstrap required, opened a new log
I20260807 00:44:09.016264 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 30932
I20260807 00:44:09.016441 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:09.016685 1490 external_mini_cluster.cc:1428] Setting key 27a132b272676dceeed14be67366408f
I20260807 00:44:09.016736 31039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188ee24c17824a93a083238c639c2b8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260807 00:44:09.016950 31039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:09.017001 31039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188ee24c17824a93a083238c639c2b8b, State: Initialized, Role: FOLLOWER
I20260807 00:44:09.017220 31039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188ee24c17824a93a083238c639c2b8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260807 00:44:09.017385 31039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:09.017446 31039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:09.017498 31039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:09.018766 31039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188ee24c17824a93a083238c639c2b8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260807 00:44:09.018913 31039 leader_election.cc:302] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 188ee24c17824a93a083238c639c2b8b; no voters:
I20260807 00:44:09.019130 31039 leader_election.cc:288] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:09.019155 31048 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:09.019292 31048 raft_consensus.cc:686] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [term 1 LEADER]: Becoming Leader. State: Replica: 188ee24c17824a93a083238c639c2b8b, State: Running, Role: LEADER
I20260807 00:44:09.019454 31039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:09.019438 31048 consensus_queue.cc:237] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188ee24c17824a93a083238c639c2b8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } }
I20260807 00:44:09.019991 31051 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 188ee24c17824a93a083238c639c2b8b. Latest consensus state: current_term: 1 leader_uuid: "188ee24c17824a93a083238c639c2b8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188ee24c17824a93a083238c639c2b8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } } }
I20260807 00:44:09.020095 31051 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:09.020663 31057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:09.021382 31057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:09.021409 31049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "188ee24c17824a93a083238c639c2b8b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "188ee24c17824a93a083238c639c2b8b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37189 } } }
I20260807 00:44:09.021538 31049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:09.023551 31057 catalog_manager.cc:1347] Generated new cluster ID: 29e4e9286adf41159195a4f81244261b
I20260807 00:44:09.023631 31057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:09.024063 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:09.017465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36651 (local address 127.0.0.1:37189)
0807 00:44:09.017677 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:09.017687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:09.017731 (+ 44us) server_negotiation.cc:407] Connection header received
0807 00:44:09.017852 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:09.017859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:09.017916 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:09.018052 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:09.018412 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:09.020025 (+ 1613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:09.021502 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:09.022105 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:09.022226 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:09.023159 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:09.023163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:09.023163 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:09.023375 (+ 212us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:09.023415 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:09.023432 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:09.023759 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:09.023807 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 00:44:09.023836 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":116,"threads_started":1}
I20260807 00:44:09.025386 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:09.036489 31057 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:09.037249 31057 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:09.040460 31045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:09.036205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59759 (local address 127.0.0.1:37189)
0807 00:44:09.036245 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:09.036247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:09.036428 (+ 181us) server_negotiation.cc:407] Connection header received
0807 00:44:09.036519 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:09.036520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:09.036537 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:09.036617 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:09.036856 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:09.037833 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:09.038906 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:09.039254 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:09.039337 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:09.040138 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:09.040141 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:09.040141 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:09.040146 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:09.040167 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:09.040172 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:09.040243 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:09.040342 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 00:44:09.040358 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":131,"server-negotiator.queue_time_us":19}
I20260807 00:44:09.051250 31057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 188ee24c17824a93a083238c639c2b8b: Generated new TSK 0
I20260807 00:44:09.051425 31057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:09.060029 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 30932
W20260807 00:44:09.074877 30910 connection.cc:535] client connection to 127.0.0.1:37189 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37189): Connection reset by peer (error 104)
2026-08-07T00:44:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:09 dist-test-slave-h6n0 krb5kdc[31090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:09 dist-test-slave-h6n0 krb5kdc[31090](info): set up 2 sockets
Aug 07 00:44:09 dist-test-slave-h6n0 krb5kdc[31090](info): commencing operation
krb5kdc: starting...
W20260807 00:44:10.330220 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:10 dist-test-slave-h6n0 krb5kdc[31090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:10.467517 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:10.551954 31110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:10.552191 31110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:10.552232 31110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:10.553359 31110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:10.553432 31110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:10.553458 31110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:10.553484 31110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:10.553514 31110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:10.554991 31110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45621
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:10.555262 31110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:10.555495 31110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:10.561093 31120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:10.561115 31117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:10.561268 31110 server_base.cc:1034] running on GCE node
W20260807 00:44:10.561122 31116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:10.561769 31110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:10.562136 31110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:10.563321 31110 hybrid_clock.cc:648] HybridClock initialized: now 1786063450563287 us; error 46 us; skew 500 ppm
Aug 07 00:44:10 dist-test-slave-h6n0 krb5kdc[31090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063450, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:10.566253 31110 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:10.566841 31110 webserver.cc:460] Webserver started at http://127.0.0.1:37987/ using document root <none> and password file <none>
I20260807 00:44:10.567134 31110 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:10.567209 31110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:10.567368 31110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:10.568614 31110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "3973edb50a4d4ee0b74637b33fe82bd8"
format_stamp: "Formatted at 2026-08-07 00:44:10 on dist-test-slave-h6n0"
server_key: "64476374568c90ad6e4fd6bbdf65ae94"
server_key_iv: "80284f8d122269875fb70e9043329110"
server_key_version: "encryptionkey@0"
I20260807 00:44:10.569047 31110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "3973edb50a4d4ee0b74637b33fe82bd8"
format_stamp: "Formatted at 2026-08-07 00:44:10 on dist-test-slave-h6n0"
server_key: "64476374568c90ad6e4fd6bbdf65ae94"
server_key_iv: "80284f8d122269875fb70e9043329110"
server_key_version: "encryptionkey@0"
I20260807 00:44:10.570576 31110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:10.571564 31132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:10.571761 31110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:10.571883 31110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "3973edb50a4d4ee0b74637b33fe82bd8"
format_stamp: "Formatted at 2026-08-07 00:44:10 on dist-test-slave-h6n0"
server_key: "64476374568c90ad6e4fd6bbdf65ae94"
server_key_iv: "80284f8d122269875fb70e9043329110"
server_key_version: "encryptionkey@0"
I20260807 00:44:10.571969 31110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:10.575855 31110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:10.578311 31110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:10.578496 31110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:10.583460 31110 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45621
I20260807 00:44:10.583508 31214 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45621 every 8 connection(s)
I20260807 00:44:10.583885 31110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:10.584986 31217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:10.588068 31217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8: Bootstrap starting.
I20260807 00:44:10.588944 31217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:10.589293 31217 log.cc:824] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:10.590188 31217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8: No bootstrap required, opened a new log
I20260807 00:44:10.592505 31217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45621 } }
I20260807 00:44:10.592670 31217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:10.592720 31217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3973edb50a4d4ee0b74637b33fe82bd8, State: Initialized, Role: FOLLOWER
I20260807 00:44:10.592869 31217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45621 } }
I20260807 00:44:10.592950 31217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:10.593000 31217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:10.593050 31217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:10.593597 31217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45621 } }
I20260807 00:44:10.593724 31217 leader_election.cc:302] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3973edb50a4d4ee0b74637b33fe82bd8; no voters:
I20260807 00:44:10.593892 31217 leader_election.cc:288] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:10.593922 31221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:10.593981 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 31110
I20260807 00:44:10.594105 31221 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [term 1 LEADER]: Becoming Leader. State: Replica: 3973edb50a4d4ee0b74637b33fe82bd8, State: Running, Role: LEADER
I20260807 00:44:10.594123 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:10.594187 31221 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45621 } }
I20260807 00:44:10.594233 31217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:10.594383 1490 external_mini_cluster.cc:1428] Setting key 4e6d495e7ca6ba874465fc91f54f84be
I20260807 00:44:10.594502 31221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45621 } } }
I20260807 00:44:10.594700 31221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:10.594882 31222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3973edb50a4d4ee0b74637b33fe82bd8. Latest consensus state: current_term: 1 leader_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3973edb50a4d4ee0b74637b33fe82bd8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45621 } } }
I20260807 00:44:10.595052 31222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:10.594997 31233 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:10.595769 31233 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:10.597638 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:10.595156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43749 (local address 127.0.0.1:45621)
0807 00:44:10.595408 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:10.595420 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:10.595645 (+ 225us) server_negotiation.cc:407] Connection header received
0807 00:44:10.595790 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:10.595797 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:10.595846 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:10.596002 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:10.597142 (+ 1140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:10.597146 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:10.597146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:10.597151 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:10.597204 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:10.597216 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:10.597296 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:10.597367 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 00:44:10.597418 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":147,"threads_started":1}
I20260807 00:44:10.598459 31233 catalog_manager.cc:1347] Generated new cluster ID: ada2bb98199d49e5b669c2450bb7a0a8
I20260807 00:44:10.598589 31233 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:10.598825 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:10.604203 31239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:10.602825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56013 (local address 127.0.0.1:45621)
0807 00:44:10.602880 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:10.602884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:10.603126 (+ 242us) server_negotiation.cc:407] Connection header received
0807 00:44:10.603203 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:10.603204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:10.603211 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:10.603260 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:10.603940 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:10.603942 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:10.603942 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:10.603946 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:10.603961 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:10.603967 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:10.604022 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:10.604106 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 00:44:10.604120 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 00:44:10.607731 31233 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:10.607908 31233 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:10.619834 31233 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3973edb50a4d4ee0b74637b33fe82bd8: Generated new TSK 0
I20260807 00:44:10.620039 31233 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:10.621628 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 31110
2026-08-07T00:44:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:10 dist-test-slave-h6n0 krb5kdc[31277](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:10 dist-test-slave-h6n0 krb5kdc[31277](info): set up 2 sockets
Aug 07 00:44:10 dist-test-slave-h6n0 krb5kdc[31277](info): commencing operation
krb5kdc: starting...
W20260807 00:44:11.865748 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.212s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:11 dist-test-slave-h6n0 krb5kdc[31277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:11.997943 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32821
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:12.087692 31298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:12.087978 31298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:12.088019 31298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:12.089206 31298 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:12.089282 31298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:12.089308 31298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:12.089329 31298 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:12.089358 31298 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:12.090910 31298 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:12.091226 31298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:12.091491 31298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:44:12.097061 31298 server_base.cc:1034] running on GCE node
W20260807 00:44:12.097033 31311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:12.097023 31308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:12.097024 31307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:12.097736 31298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:12.098094 31298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:12.099290 31298 hybrid_clock.cc:648] HybridClock initialized: now 1786063452099271 us; error 38 us; skew 500 ppm
Aug 07 00:44:12 dist-test-slave-h6n0 krb5kdc[31277](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063452, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:12.102312 31298 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:12.102873 31298 webserver.cc:460] Webserver started at http://127.0.0.1:44449/ using document root <none> and password file <none>
I20260807 00:44:12.103174 31298 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:12.103232 31298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:12.103405 31298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:12.104592 31298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "f770cd2eed6f46b1a069a56cf8940b31"
format_stamp: "Formatted at 2026-08-07 00:44:12 on dist-test-slave-h6n0"
server_key: "14daa947e03b70be2a8501259251eca1"
server_key_iv: "038426b6d4abfb6528814f26baa4c77f"
server_key_version: "encryptionkey@0"
I20260807 00:44:12.104980 31298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "f770cd2eed6f46b1a069a56cf8940b31"
format_stamp: "Formatted at 2026-08-07 00:44:12 on dist-test-slave-h6n0"
server_key: "14daa947e03b70be2a8501259251eca1"
server_key_iv: "038426b6d4abfb6528814f26baa4c77f"
server_key_version: "encryptionkey@0"
I20260807 00:44:12.106475 31298 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:12.107436 31323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:12.107734 31298 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:12.107862 31298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "f770cd2eed6f46b1a069a56cf8940b31"
format_stamp: "Formatted at 2026-08-07 00:44:12 on dist-test-slave-h6n0"
server_key: "14daa947e03b70be2a8501259251eca1"
server_key_iv: "038426b6d4abfb6528814f26baa4c77f"
server_key_version: "encryptionkey@0"
I20260807 00:44:12.107942 31298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:12.117869 31298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:12.120013 31298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:12.120206 31298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:12.125469 31298 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32821
I20260807 00:44:12.125528 31400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32821 every 8 connection(s)
I20260807 00:44:12.125924 31298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:12.127120 31401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:12.129490 31401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31: Bootstrap starting.
I20260807 00:44:12.130038 31401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:12.130278 31401 log.cc:824] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:12.131086 31401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31: No bootstrap required, opened a new log
I20260807 00:44:12.132632 31401 raft_consensus.cc:348] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f770cd2eed6f46b1a069a56cf8940b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260807 00:44:12.132771 31401 raft_consensus.cc:374] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:12.132793 31401 raft_consensus.cc:729] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f770cd2eed6f46b1a069a56cf8940b31, State: Initialized, Role: FOLLOWER
I20260807 00:44:12.132895 31401 consensus_queue.cc:260] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f770cd2eed6f46b1a069a56cf8940b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260807 00:44:12.132961 31401 raft_consensus.cc:388] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:12.132982 31401 raft_consensus.cc:482] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:12.133068 31401 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:12.133579 31401 raft_consensus.cc:504] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f770cd2eed6f46b1a069a56cf8940b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260807 00:44:12.133669 31401 leader_election.cc:302] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f770cd2eed6f46b1a069a56cf8940b31; no voters:
I20260807 00:44:12.133839 31401 leader_election.cc:288] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:12.133925 31405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:12.134140 31401 sys_catalog.cc:564] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:12.134225 31405 raft_consensus.cc:686] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [term 1 LEADER]: Becoming Leader. State: Replica: f770cd2eed6f46b1a069a56cf8940b31, State: Running, Role: LEADER
I20260807 00:44:12.134460 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 31298
I20260807 00:44:12.134438 31405 consensus_queue.cc:237] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f770cd2eed6f46b1a069a56cf8940b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } }
I20260807 00:44:12.134624 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:12.134866 1490 external_mini_cluster.cc:1428] Setting key 3ef0836dca115a9400af2b0fb87bc68b
I20260807 00:44:12.135066 31410 sys_catalog.cc:455] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f770cd2eed6f46b1a069a56cf8940b31. Latest consensus state: current_term: 1 leader_uuid: "f770cd2eed6f46b1a069a56cf8940b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f770cd2eed6f46b1a069a56cf8940b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260807 00:44:12.135170 31410 sys_catalog.cc:458] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:12.135233 31409 sys_catalog.cc:455] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f770cd2eed6f46b1a069a56cf8940b31" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f770cd2eed6f46b1a069a56cf8940b31" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32821 } } }
I20260807 00:44:12.135350 31409 sys_catalog.cc:458] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:12.135452 31422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:12.136222 31422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:12.138464 31422 catalog_manager.cc:1347] Generated new cluster ID: e75314819aca48ef98bc471a187303a9
I20260807 00:44:12.138535 31422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:44:12 dist-test-slave-h6n0 krb5kdc[31277](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:44:12.148784 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:12.135604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35581 (local address 127.0.0.1:32821)
0807 00:44:12.136403 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:12.136424 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:12.136467 (+ 43us) server_negotiation.cc:407] Connection header received
0807 00:44:12.136602 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:12.136610 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:12.136667 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:12.136901 (+ 234us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:12.138005 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:12.139571 (+ 1566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:12.140676 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:12.141385 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:12.141514 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:12.144745 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:12.144750 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:12.144750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:12.144765 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:12.146941 (+ 2176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:12.147445 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:12.147448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:12.147448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:12.147551 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:12.147787 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:12.147788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:12.147788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:12.148110 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:12.148278 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:12.148412 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 00:44:12.148555 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":78,"threads_started":1}
I20260807 00:44:12.149839 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:12.159271 31424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:12.151659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52499 (local address 127.0.0.1:32821)
0807 00:44:12.151698 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:12.151699 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:12.151873 (+ 174us) server_negotiation.cc:407] Connection header received
0807 00:44:12.151926 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:12.151927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:12.151937 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:12.152009 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:12.152687 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:12.153459 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:12.154615 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:12.154930 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:12.154998 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:12.156478 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:12.156479 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:12.156479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:12.156485 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:12.158045 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:12.158460 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:12.158461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:12.158462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:12.158517 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:12.158760 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:12.158761 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:12.158762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:12.158876 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:12.158967 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:12.159122 (+ 155us) server_negotiation.cc:299] Negotiation successful
0807 00:44:12.159192 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 00:44:12.161660 31422 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:12.162189 31422 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:12.168239 31422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f770cd2eed6f46b1a069a56cf8940b31: Generated new TSK 0
I20260807 00:44:12.168483 31422 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:12.172230 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 31298
2026-08-07T00:44:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1548 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:12 dist-test-slave-h6n0 krb5kdc[31451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:12 dist-test-slave-h6n0 krb5kdc[31451](info): set up 2 sockets
Aug 07 00:44:12 dist-test-slave-h6n0 krb5kdc[31451](info): commencing operation
krb5kdc: starting...
W20260807 00:44:13.422758 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.221s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:13 dist-test-slave-h6n0 krb5kdc[31451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:13.558113 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37461
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37461
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:13.643774 31470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:13.644001 31470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:13.644024 31470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:13.645077 31470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:13.645117 31470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:13.645131 31470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:13.645144 31470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:13.645156 31470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:13.646669 31470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37461
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37461
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:13.646862 31470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:13.647022 31470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:13.652501 31480 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:13.652458 31486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:13.652483 31477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:13.652993 31470 server_base.cc:1034] running on GCE node
I20260807 00:44:13.653263 31470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:13.653587 31470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:13.654783 31470 hybrid_clock.cc:648] HybridClock initialized: now 1786063453654760 us; error 45 us; skew 500 ppm
Aug 07 00:44:13 dist-test-slave-h6n0 krb5kdc[31451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063453, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:13.662129 31470 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:13.662694 31470 webserver.cc:460] Webserver started at http://127.0.0.1:35251/ using document root <none> and password file <none>
I20260807 00:44:13.662957 31470 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:13.663029 31470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:13.663174 31470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:13.664283 31470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "56a47629b21345c7a15ca227985ee515"
format_stamp: "Formatted at 2026-08-07 00:44:13 on dist-test-slave-h6n0"
server_key: "da984463c0ba048f57d2862d0bbfbf60"
server_key_iv: "4ec1e3d630734b3ddafbf51800f44413"
server_key_version: "encryptionkey@0"
I20260807 00:44:13.664661 31470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "56a47629b21345c7a15ca227985ee515"
format_stamp: "Formatted at 2026-08-07 00:44:13 on dist-test-slave-h6n0"
server_key: "da984463c0ba048f57d2862d0bbfbf60"
server_key_iv: "4ec1e3d630734b3ddafbf51800f44413"
server_key_version: "encryptionkey@0"
I20260807 00:44:13.666201 31470 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:13.667500 31496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:13.667840 31470 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:13.667938 31470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "56a47629b21345c7a15ca227985ee515"
format_stamp: "Formatted at 2026-08-07 00:44:13 on dist-test-slave-h6n0"
server_key: "da984463c0ba048f57d2862d0bbfbf60"
server_key_iv: "4ec1e3d630734b3ddafbf51800f44413"
server_key_version: "encryptionkey@0"
I20260807 00:44:13.668046 31470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:13.691376 31470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:13.693773 31470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:13.693943 31470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:13.698735 31470 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37461
I20260807 00:44:13.698807 31568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37461 every 8 connection(s)
I20260807 00:44:13.699200 31470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:13.700459 31569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:13.703334 31569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515: Bootstrap starting.
I20260807 00:44:13.704041 31569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:13.704213 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 31470
I20260807 00:44:13.704363 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:13.704457 31569 log.cc:824] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:13.704645 1490 external_mini_cluster.cc:1428] Setting key f0b26e49ea902ea57df8ac072195954a
I20260807 00:44:13.705576 31569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515: No bootstrap required, opened a new log
I20260807 00:44:13.709174 31569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a47629b21345c7a15ca227985ee515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260807 00:44:13.709406 31569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:13.709506 31569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56a47629b21345c7a15ca227985ee515, State: Initialized, Role: FOLLOWER
I20260807 00:44:13.709725 31569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a47629b21345c7a15ca227985ee515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260807 00:44:13.709838 31569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:13.709934 31569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:13.710004 31569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:13.710669 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:13.705276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44613 (local address 127.0.0.1:37461)
0807 00:44:13.705476 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:13.705488 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:13.705529 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:44:13.706125 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:13.706133 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:13.706198 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:13.706403 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:13.706628 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:13.707626 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:13.708463 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:13.708921 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:13.709061 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:13.709821 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:13.709826 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:13.709826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:13.710074 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:13.710119 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:13.710146 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:13.710220 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:13.710300 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 00:44:13.710380 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":84,"threads_started":1}
I20260807 00:44:13.711337 31569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a47629b21345c7a15ca227985ee515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260807 00:44:13.711485 31569 leader_election.cc:302] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56a47629b21345c7a15ca227985ee515; no voters:
I20260807 00:44:13.711699 31569 leader_election.cc:288] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:13.711817 31580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:13.711815 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:13.712008 31580 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [term 1 LEADER]: Becoming Leader. State: Replica: 56a47629b21345c7a15ca227985ee515, State: Running, Role: LEADER
I20260807 00:44:13.712100 31569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:13.712128 31580 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a47629b21345c7a15ca227985ee515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } }
I20260807 00:44:13.712572 31580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56a47629b21345c7a15ca227985ee515. Latest consensus state: current_term: 1 leader_uuid: "56a47629b21345c7a15ca227985ee515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a47629b21345c7a15ca227985ee515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } } }
I20260807 00:44:13.712678 31580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:13.712751 31581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56a47629b21345c7a15ca227985ee515" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56a47629b21345c7a15ca227985ee515" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37461 } } }
I20260807 00:44:13.712939 31581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515 [sys.catalog]: This master's current role is: LEADER
W20260807 00:44:13.714962 31607 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:13.715036 31607 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:13.715167 31613 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:13.715366 31613 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:13.717813 31613 catalog_manager.cc:1347] Generated new cluster ID: e90d7d366c7d4649b494b8deb0d78ebc
I20260807 00:44:13.717931 31613 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:13.718166 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:13.714029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40841 (local address 127.0.0.1:37461)
0807 00:44:13.714091 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:13.714095 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:13.714266 (+ 171us) server_negotiation.cc:407] Connection header received
0807 00:44:13.714374 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:13.714376 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:13.714392 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:13.714478 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:13.714711 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:13.715467 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:13.716425 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:13.716736 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:13.716821 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:13.717719 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:13.717720 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:13.717721 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:13.717726 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:13.717749 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:13.717754 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:13.717799 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:13.718056 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 00:44:13.718086 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 00:44:13.723208 31613 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:13.723878 31613 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:13.727317 31613 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56a47629b21345c7a15ca227985ee515: Generated new TSK 0
I20260807 00:44:13.727509 31613 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:13.730372 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 31470
2026-08-07T00:44:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:13 dist-test-slave-h6n0 krb5kdc[31621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:13 dist-test-slave-h6n0 krb5kdc[31621](info): set up 2 sockets
Aug 07 00:44:13 dist-test-slave-h6n0 krb5kdc[31621](info): commencing operation
krb5kdc: starting...
W20260807 00:44:14.978106 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:15 dist-test-slave-h6n0 krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063455, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:15.109097 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:15.199280 31644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:15.199540 31644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:15.199577 31644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:15.200708 31644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:15.200778 31644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:15.200804 31644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:15.200826 31644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:15.200855 31644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:15.202443 31644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:15.202772 31644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:15.203023 31644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:15.208521 31657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:15.208539 31655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:15.208737 31644 server_base.cc:1034] running on GCE node
W20260807 00:44:15.208518 31653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:15.209225 31644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:15.209581 31644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:15.210767 31644 hybrid_clock.cc:648] HybridClock initialized: now 1786063455210751 us; error 44 us; skew 500 ppm
Aug 07 00:44:15 dist-test-slave-h6n0 krb5kdc[31621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063455, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:15.213677 31644 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:15.214211 31644 webserver.cc:460] Webserver started at http://127.0.0.1:38351/ using document root <none> and password file <none>
I20260807 00:44:15.214483 31644 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:15.214581 31644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:15.214754 31644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:15.215939 31644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "421aa03d62b44cbba3882a6ba142e52e"
format_stamp: "Formatted at 2026-08-07 00:44:15 on dist-test-slave-h6n0"
server_key: "78b963e69fc038a60de3734b1a209bd3"
server_key_iv: "20f4993f50223d45a9e4ecccc26df126"
server_key_version: "encryptionkey@0"
I20260807 00:44:15.216343 31644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "421aa03d62b44cbba3882a6ba142e52e"
format_stamp: "Formatted at 2026-08-07 00:44:15 on dist-test-slave-h6n0"
server_key: "78b963e69fc038a60de3734b1a209bd3"
server_key_iv: "20f4993f50223d45a9e4ecccc26df126"
server_key_version: "encryptionkey@0"
I20260807 00:44:15.217794 31644 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:15.218695 31664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:15.218912 31644 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:15.219027 31644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "421aa03d62b44cbba3882a6ba142e52e"
format_stamp: "Formatted at 2026-08-07 00:44:15 on dist-test-slave-h6n0"
server_key: "78b963e69fc038a60de3734b1a209bd3"
server_key_iv: "20f4993f50223d45a9e4ecccc26df126"
server_key_version: "encryptionkey@0"
I20260807 00:44:15.219112 31644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:15.222733 31644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:15.224701 31644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:15.224869 31644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:15.230000 31644 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33099
I20260807 00:44:15.230042 31758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33099 every 8 connection(s)
I20260807 00:44:15.230420 31644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:15.231549 31759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:15.233913 31759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e: Bootstrap starting.
I20260807 00:44:15.234601 31759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:15.234742 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 31644
I20260807 00:44:15.234871 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:15.234970 31759 log.cc:824] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:15.235113 1490 external_mini_cluster.cc:1428] Setting key 529349ccb5ea128c27c95961300ab1f9
I20260807 00:44:15.235802 31759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e: No bootstrap required, opened a new log
I20260807 00:44:15.237939 31763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:15.235672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45107 (local address 127.0.0.1:33099)
0807 00:44:15.235896 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:15.235907 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:15.235945 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:44:15.236536 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:15.236544 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:15.236571 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:15.236744 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:15.237457 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:15.237461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:15.237462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:15.237466 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:15.237518 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:15.237530 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:15.237632 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:15.237703 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 00:44:15.237753 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":86,"threads_started":1}
I20260807 00:44:15.238889 31759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421aa03d62b44cbba3882a6ba142e52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33099 } }
I20260807 00:44:15.239066 31759 raft_consensus.cc:374] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:15.239120 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:15.239146 31759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 421aa03d62b44cbba3882a6ba142e52e, State: Initialized, Role: FOLLOWER
I20260807 00:44:15.239287 31759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421aa03d62b44cbba3882a6ba142e52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33099 } }
I20260807 00:44:15.239382 31759 raft_consensus.cc:388] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:15.239425 31759 raft_consensus.cc:482] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:15.239473 31759 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:15.240046 31759 raft_consensus.cc:504] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421aa03d62b44cbba3882a6ba142e52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33099 } }
I20260807 00:44:15.240165 31759 leader_election.cc:302] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 421aa03d62b44cbba3882a6ba142e52e; no voters:
I20260807 00:44:15.240404 31759 leader_election.cc:288] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:15.240432 31766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:15.240715 31766 raft_consensus.cc:686] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [term 1 LEADER]: Becoming Leader. State: Replica: 421aa03d62b44cbba3882a6ba142e52e, State: Running, Role: LEADER
I20260807 00:44:15.240772 31759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:15.240907 31766 consensus_queue.cc:237] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421aa03d62b44cbba3882a6ba142e52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33099 } }
I20260807 00:44:15.241317 31773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "421aa03d62b44cbba3882a6ba142e52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421aa03d62b44cbba3882a6ba142e52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33099 } } }
I20260807 00:44:15.241304 31766 sys_catalog.cc:455] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 421aa03d62b44cbba3882a6ba142e52e. Latest consensus state: current_term: 1 leader_uuid: "421aa03d62b44cbba3882a6ba142e52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "421aa03d62b44cbba3882a6ba142e52e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33099 } } }
I20260807 00:44:15.241415 31766 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:15.241483 31773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:15.242182 31784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:15.242759 31784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:15.243414 31763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:15.240993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47563 (local address 127.0.0.1:33099)
0807 00:44:15.241028 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:15.241030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:15.241855 (+ 825us) server_negotiation.cc:407] Connection header received
0807 00:44:15.241879 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:15.241880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:15.241885 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:15.241961 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:15.243008 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:15.243010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:15.243010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:15.243015 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:15.243033 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:15.243037 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:15.243108 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:15.243326 (+ 218us) server_negotiation.cc:299] Negotiation successful
0807 00:44:15.243344 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260807 00:44:15.244863 31784 catalog_manager.cc:1347] Generated new cluster ID: af6ab912e3cb4666a423095efb9cc8ad
I20260807 00:44:15.244922 31784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:15.259737 31784 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:15.260013 31784 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:15.264591 31784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 421aa03d62b44cbba3882a6ba142e52e: Generated new TSK 0
I20260807 00:44:15.264788 31784 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:15.272961 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 31644
2026-08-07T00:44:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1546 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:15 dist-test-slave-h6n0 krb5kdc[31807](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:15 dist-test-slave-h6n0 krb5kdc[31807](info): set up 2 sockets
Aug 07 00:44:15 dist-test-slave-h6n0 krb5kdc[31807](info): commencing operation
krb5kdc: starting...
W20260807 00:44:16.527256 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:16 dist-test-slave-h6n0 krb5kdc[31807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:16.659749 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44669
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:16.752307 31827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:16.752528 31827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:16.752552 31827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:16.753712 31827 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:16.753762 31827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:16.753777 31827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:16.753789 31827 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:16.753801 31827 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:16.755441 31827 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:16.755733 31827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:16.756008 31827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:16.761526 31839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:16.761545 31843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:16.761545 31845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:16.761795 31827 server_base.cc:1034] running on GCE node
I20260807 00:44:16.762092 31827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:16.762459 31827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:16.763615 31827 hybrid_clock.cc:648] HybridClock initialized: now 1786063456763593 us; error 50 us; skew 500 ppm
Aug 07 00:44:16 dist-test-slave-h6n0 krb5kdc[31807](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063456, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:16.766407 31827 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:16.766968 31827 webserver.cc:460] Webserver started at http://127.0.0.1:32897/ using document root <none> and password file <none>
I20260807 00:44:16.767249 31827 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:16.767318 31827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:16.767474 31827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:16.768682 31827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "4e9bd3fd513b473c9fe05011e701101a"
format_stamp: "Formatted at 2026-08-07 00:44:16 on dist-test-slave-h6n0"
server_key: "c5736cda4116b3953e9940e86cd50162"
server_key_iv: "20e851172bb0fd3e99b995085f1d5527"
server_key_version: "encryptionkey@0"
I20260807 00:44:16.769079 31827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "4e9bd3fd513b473c9fe05011e701101a"
format_stamp: "Formatted at 2026-08-07 00:44:16 on dist-test-slave-h6n0"
server_key: "c5736cda4116b3953e9940e86cd50162"
server_key_iv: "20e851172bb0fd3e99b995085f1d5527"
server_key_version: "encryptionkey@0"
I20260807 00:44:16.770669 31827 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:44:16.771562 31854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:16.771802 31827 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:16.771922 31827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "4e9bd3fd513b473c9fe05011e701101a"
format_stamp: "Formatted at 2026-08-07 00:44:16 on dist-test-slave-h6n0"
server_key: "c5736cda4116b3953e9940e86cd50162"
server_key_iv: "20e851172bb0fd3e99b995085f1d5527"
server_key_version: "encryptionkey@0"
I20260807 00:44:16.772001 31827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:16.793879 31827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:16.796283 31827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:16.796464 31827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:16.802084 31827 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44669
I20260807 00:44:16.802161 31939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44669 every 8 connection(s)
I20260807 00:44:16.802651 31827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:16.803905 31940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:16.805982 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 31827
I20260807 00:44:16.806137 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:16.806396 1490 external_mini_cluster.cc:1428] Setting key ef5946f06b3c99bf14b36ac246ff2b48
I20260807 00:44:16.806573 31940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a: Bootstrap starting.
I20260807 00:44:16.807363 31940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:16.807754 31940 log.cc:824] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:16.808652 31940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a: No bootstrap required, opened a new log
I20260807 00:44:16.810688 31940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9bd3fd513b473c9fe05011e701101a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44669 } }
I20260807 00:44:16.810846 31940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:16.810895 31940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e9bd3fd513b473c9fe05011e701101a, State: Initialized, Role: FOLLOWER
I20260807 00:44:16.811059 31940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9bd3fd513b473c9fe05011e701101a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44669 } }
I20260807 00:44:16.811158 31940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:16.811201 31940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:16.811251 31940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:16.812038 31940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9bd3fd513b473c9fe05011e701101a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44669 } }
I20260807 00:44:16.812188 31940 leader_election.cc:302] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e9bd3fd513b473c9fe05011e701101a; no voters:
I20260807 00:44:16.812369 31940 leader_election.cc:288] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:16.812410 31947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:16.812692 31947 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [term 1 LEADER]: Becoming Leader. State: Replica: 4e9bd3fd513b473c9fe05011e701101a, State: Running, Role: LEADER
I20260807 00:44:16.812669 31940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:16.812893 31947 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9bd3fd513b473c9fe05011e701101a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44669 } }
I20260807 00:44:16.813632 31949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e9bd3fd513b473c9fe05011e701101a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9bd3fd513b473c9fe05011e701101a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44669 } } }
I20260807 00:44:16.813761 31949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:16.814033 31959 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:16.814087 31950 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e9bd3fd513b473c9fe05011e701101a. Latest consensus state: current_term: 1 leader_uuid: "4e9bd3fd513b473c9fe05011e701101a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e9bd3fd513b473c9fe05011e701101a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44669 } } }
I20260807 00:44:16.814191 31950 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a [sys.catalog]: This master's current role is: LEADER
Aug 07 00:44:16 dist-test-slave-h6n0 krb5kdc[31807](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:44:16.814692 31959 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:16.816696 31959 catalog_manager.cc:1347] Generated new cluster ID: 89de32f9e63f489aadb5cf833e88939f
I20260807 00:44:16.816761 31959 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:16.819741 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:16.807180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:52417 (local address 127.0.0.1:44669)
0807 00:44:16.807607 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:16.807622 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:16.807674 (+ 52us) server_negotiation.cc:407] Connection header received
0807 00:44:16.808515 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:16.808525 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:16.808586 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:16.808770 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:16.809481 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:16.810466 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:16.811453 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:16.811920 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:16.812069 (+ 149us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:16.815415 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:16.815420 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:16.815421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:16.815436 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:16.817850 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:16.818339 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:16.818342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:16.818342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:16.818454 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:16.818770 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:16.818771 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:16.818771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:16.819111 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:16.819290 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:16.819406 (+ 116us) server_negotiation.cc:299] Negotiation successful
0807 00:44:16.819511 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":92,"threads_started":1}
I20260807 00:44:16.821306 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:16.834908 31945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:16.826861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:46541 (local address 127.0.0.1:44669)
0807 00:44:16.826909 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:16.826912 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:16.827116 (+ 204us) server_negotiation.cc:407] Connection header received
0807 00:44:16.827179 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:16.827180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:16.827192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:16.827252 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:16.827925 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:16.828724 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:16.829797 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:16.830122 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:16.830191 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:16.831758 (+ 1567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:16.831760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:16.831760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:16.831767 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:16.833494 (+ 1727us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:16.833979 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:16.833980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:16.833981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:16.834040 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:16.834297 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:16.834297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:16.834298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:16.834427 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:16.834572 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:16.834740 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:44:16.834807 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:44:16.842298 31959 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:16.842994 31959 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:16.847450 31959 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e9bd3fd513b473c9fe05011e701101a: Generated new TSK 0
I20260807 00:44:16.847630 31959 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:16.855880 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 31827
W20260807 00:44:16.868134 31804 connection.cc:535] client connection to 127.0.0.1:44669 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44669): Connection reset by peer (error 104)
2026-08-07T00:44:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1582 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:16 dist-test-slave-h6n0 krb5kdc[31978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:16 dist-test-slave-h6n0 krb5kdc[31978](info): set up 2 sockets
Aug 07 00:44:16 dist-test-slave-h6n0 krb5kdc[31978](info): commencing operation
krb5kdc: starting...
W20260807 00:44:18.196422 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.314s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:18 dist-test-slave-h6n0 krb5kdc[31978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:18.335224 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45003
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:18.429872 32002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:18.430167 32002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:18.430241 32002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:18.431511 32002 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:18.431586 32002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:18.431613 32002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:18.431646 32002 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:18.431674 32002 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:18.433382 32002 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:18.433707 32002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:18.433976 32002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:18.439565 32010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:18.439782 32002 server_base.cc:1034] running on GCE node
W20260807 00:44:18.439565 32014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:18.439548 32012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:18.440204 32002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:18.440583 32002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:18.441767 32002 hybrid_clock.cc:648] HybridClock initialized: now 1786063458441755 us; error 64 us; skew 500 ppm
Aug 07 00:44:18 dist-test-slave-h6n0 krb5kdc[31978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063458, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:18.444674 32002 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:18.445151 32002 webserver.cc:460] Webserver started at http://127.0.0.1:42815/ using document root <none> and password file <none>
I20260807 00:44:18.445433 32002 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:18.445504 32002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:18.445663 32002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:18.446834 32002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "b1aa68168cc84a73b9681977b1486f68"
format_stamp: "Formatted at 2026-08-07 00:44:18 on dist-test-slave-h6n0"
server_key: "400ebfcb5d4854b02e59050d12d22d41"
server_key_iv: "5d44fe7090186b84ab1e8f899135a4ef"
server_key_version: "encryptionkey@0"
I20260807 00:44:18.447209 32002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "b1aa68168cc84a73b9681977b1486f68"
format_stamp: "Formatted at 2026-08-07 00:44:18 on dist-test-slave-h6n0"
server_key: "400ebfcb5d4854b02e59050d12d22d41"
server_key_iv: "5d44fe7090186b84ab1e8f899135a4ef"
server_key_version: "encryptionkey@0"
I20260807 00:44:18.448649 32002 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260807 00:44:18.449502 32023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:18.449733 32002 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:18.449833 32002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "b1aa68168cc84a73b9681977b1486f68"
format_stamp: "Formatted at 2026-08-07 00:44:18 on dist-test-slave-h6n0"
server_key: "400ebfcb5d4854b02e59050d12d22d41"
server_key_iv: "5d44fe7090186b84ab1e8f899135a4ef"
server_key_version: "encryptionkey@0"
I20260807 00:44:18.449921 32002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:18.477038 32002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:18.479635 32002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:18.479803 32002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:18.484565 32002 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45003
I20260807 00:44:18.484658 32102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45003 every 8 connection(s)
I20260807 00:44:18.485001 32002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:18.486223 32103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:18.489295 32103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68: Bootstrap starting.
I20260807 00:44:18.489982 32103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:18.490314 32103 log.cc:824] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:18.491266 32103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68: No bootstrap required, opened a new log
I20260807 00:44:18.491842 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 32002
I20260807 00:44:18.491976 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:18.492224 1490 external_mini_cluster.cc:1428] Setting key 6a2495e177627e9a04732f2738f8076b
I20260807 00:44:18.493820 32103 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aa68168cc84a73b9681977b1486f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45003 } }
I20260807 00:44:18.494120 32103 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:18.494228 32103 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1aa68168cc84a73b9681977b1486f68, State: Initialized, Role: FOLLOWER
I20260807 00:44:18.494457 32103 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aa68168cc84a73b9681977b1486f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45003 } }
I20260807 00:44:18.494616 32103 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:18.494832 32103 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:18.494927 32103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:18.495570 32103 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aa68168cc84a73b9681977b1486f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45003 } }
I20260807 00:44:18.495697 32103 leader_election.cc:302] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1aa68168cc84a73b9681977b1486f68; no voters:
I20260807 00:44:18.495913 32103 leader_election.cc:288] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:18.495966 32111 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:18.496222 32111 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [term 1 LEADER]: Becoming Leader. State: Replica: b1aa68168cc84a73b9681977b1486f68, State: Running, Role: LEADER
I20260807 00:44:18.496361 32103 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:18.496402 32111 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aa68168cc84a73b9681977b1486f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45003 } }
I20260807 00:44:18.496874 32112 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1aa68168cc84a73b9681977b1486f68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aa68168cc84a73b9681977b1486f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45003 } } }
I20260807 00:44:18.496865 32111 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1aa68168cc84a73b9681977b1486f68. Latest consensus state: current_term: 1 leader_uuid: "b1aa68168cc84a73b9681977b1486f68" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1aa68168cc84a73b9681977b1486f68" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45003 } } }
I20260807 00:44:18.496969 32112 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:18.496973 32111 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:18.497269 32117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:18.497933 32117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:18.500275 32117 catalog_manager.cc:1347] Generated new cluster ID: c78a6362fe024e7cbb7310a30771f0dc
I20260807 00:44:18.500353 32117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:18.500586 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:18.492972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:43745 (local address 127.0.0.1:45003)
0807 00:44:18.493336 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:18.493345 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:18.493384 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:44:18.493809 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:18.493816 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:18.493868 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:18.493993 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:18.494293 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:18.495757 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:18.497185 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:18.498066 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:18.498223 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:18.499446 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:18.499451 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:18.499451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:18.499683 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:18.499728 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:18.499745 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:18.499851 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:18.500194 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 00:44:18.500258 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":124,"threads_started":1}
I20260807 00:44:18.501668 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:18.509886 32110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:18.503263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:58221 (local address 127.0.0.1:45003)
0807 00:44:18.503311 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:18.503313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:18.506890 (+ 3577us) server_negotiation.cc:407] Connection header received
0807 00:44:18.506970 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:18.506971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:18.506984 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:18.507027 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:18.507202 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:18.507737 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:18.508496 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:18.508740 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:18.508830 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:18.509589 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:18.509591 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:18.509591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:18.509596 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:18.509612 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:18.509617 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:18.509675 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:18.509751 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 00:44:18.509763 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:44:18.519806 32117 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:18.520534 32117 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:18.550297 32117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1aa68168cc84a73b9681977b1486f68: Generated new TSK 0
I20260807 00:44:18.550487 32117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:18.559285 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 32002
W20260807 00:44:18.574304 31976 connection.cc:535] client connection to 127.0.0.1:45003 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45003): Connection reset by peer (error 104)
2026-08-07T00:44:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1706 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:18 dist-test-slave-h6n0 krb5kdc[32152](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:18 dist-test-slave-h6n0 krb5kdc[32152](info): set up 2 sockets
Aug 07 00:44:18 dist-test-slave-h6n0 krb5kdc[32152](info): commencing operation
krb5kdc: starting...
W20260807 00:44:20.271528 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.683s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:20 dist-test-slave-h6n0 krb5kdc[32152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:20.498859 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44213
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37595
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44213
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:20.653743 32174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:20.654045 32174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:20.654081 32174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:20.656069 32174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:20.656158 32174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:20.656189 32174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:20.656224 32174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:20.656246 32174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:20.658934 32174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37595
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44213
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:20.659425 32174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:20.659701 32174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:20.667572 32186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:20.667513 32185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:20.667706 32189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:20.668274 32174 server_base.cc:1034] running on GCE node
I20260807 00:44:20.668709 32174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:20.669270 32174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:20.670491 32174 hybrid_clock.cc:648] HybridClock initialized: now 1786063460670454 us; error 69 us; skew 500 ppm
Aug 07 00:44:20 dist-test-slave-h6n0 krb5kdc[32152](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:20.675693 32174 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:20.676477 32174 webserver.cc:460] Webserver started at http://127.0.0.1:44211/ using document root <none> and password file <none>
I20260807 00:44:20.676991 32174 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:20.677120 32174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:20.677383 32174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:20.679458 32174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "5764e46f0e5148d79705a945f7689635"
format_stamp: "Formatted at 2026-08-07 00:44:20 on dist-test-slave-h6n0"
server_key: "108bf01955678456f0f2900691f1bf66"
server_key_iv: "bd8654febb962afd75caa341a0f529ae"
server_key_version: "encryptionkey@0"
I20260807 00:44:20.680192 32174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "5764e46f0e5148d79705a945f7689635"
format_stamp: "Formatted at 2026-08-07 00:44:20 on dist-test-slave-h6n0"
server_key: "108bf01955678456f0f2900691f1bf66"
server_key_iv: "bd8654febb962afd75caa341a0f529ae"
server_key_version: "encryptionkey@0"
I20260807 00:44:20.683158 32174 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:44:20.684638 32197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:20.684980 32174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:20.685174 32174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "5764e46f0e5148d79705a945f7689635"
format_stamp: "Formatted at 2026-08-07 00:44:20 on dist-test-slave-h6n0"
server_key: "108bf01955678456f0f2900691f1bf66"
server_key_iv: "bd8654febb962afd75caa341a0f529ae"
server_key_version: "encryptionkey@0"
I20260807 00:44:20.685312 32174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:20.693311 32174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:20.697227 32174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:20.697505 32174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:20.706797 32174 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44213
I20260807 00:44:20.706984 32271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44213 every 8 connection(s)
I20260807 00:44:20.707564 32174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:20.709403 32274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:20.711097 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 32174
I20260807 00:44:20.711251 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:20.711732 1490 external_mini_cluster.cc:1428] Setting key 3aa1da337f4dae7cdad8ba2cbbdb954c
I20260807 00:44:20.716048 32274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635: Bootstrap starting.
I20260807 00:44:20.716626 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:20.712672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:33613 (local address 127.0.0.1:44213)
0807 00:44:20.713003 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:20.713021 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:20.713074 (+ 53us) server_negotiation.cc:407] Connection header received
0807 00:44:20.713956 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:20.713966 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:20.714018 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:20.714266 (+ 248us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:20.715701 (+ 1435us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:20.715706 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:20.715706 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:20.715711 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:20.715788 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:20.715810 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:20.715914 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:20.716194 (+ 280us) server_negotiation.cc:299] Negotiation successful
0807 00:44:20.716333 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":183,"threads_started":1}
I20260807 00:44:20.717424 32274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:20.718118 32274 log.cc:824] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:20.719089 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:20.720139 32274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635: No bootstrap required, opened a new log
I20260807 00:44:20.724397 32278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:20.721986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:58459 (local address 127.0.0.1:44213)
0807 00:44:20.722060 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:20.722066 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:20.722295 (+ 229us) server_negotiation.cc:407] Connection header received
0807 00:44:20.722489 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:20.722491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:20.722498 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:20.722703 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:20.723989 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:20.723991 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:20.723991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:20.723997 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:20.724019 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:20.724024 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:20.724153 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:20.724261 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 00:44:20.724289 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:44:20.724396 32274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5764e46f0e5148d79705a945f7689635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44213 } }
I20260807 00:44:20.724885 32274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:20.725018 32274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5764e46f0e5148d79705a945f7689635, State: Initialized, Role: FOLLOWER
I20260807 00:44:20.725257 32274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5764e46f0e5148d79705a945f7689635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44213 } }
I20260807 00:44:20.725469 32274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:20.725561 32274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:20.725680 32274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:20.726681 32274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5764e46f0e5148d79705a945f7689635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44213 } }
I20260807 00:44:20.726884 32274 leader_election.cc:302] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5764e46f0e5148d79705a945f7689635; no voters:
I20260807 00:44:20.727180 32274 leader_election.cc:288] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:20.727250 32290 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:20.727591 32290 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [term 1 LEADER]: Becoming Leader. State: Replica: 5764e46f0e5148d79705a945f7689635, State: Running, Role: LEADER
I20260807 00:44:20.727711 32274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:20.727880 32290 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5764e46f0e5148d79705a945f7689635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44213 } }
I20260807 00:44:20.728648 32293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5764e46f0e5148d79705a945f7689635. Latest consensus state: current_term: 1 leader_uuid: "5764e46f0e5148d79705a945f7689635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5764e46f0e5148d79705a945f7689635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44213 } } }
I20260807 00:44:20.728871 32293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:20.728952 32292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5764e46f0e5148d79705a945f7689635" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5764e46f0e5148d79705a945f7689635" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44213 } } }
I20260807 00:44:20.729068 32292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:20.729509 32302 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:20.731036 32302 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:20.734169 32302 catalog_manager.cc:1347] Generated new cluster ID: ee7b3265542f4087b02642f70897bae3
I20260807 00:44:20.734283 32302 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:20.751375 32302 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:20.751688 32302 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:20.762166 32302 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5764e46f0e5148d79705a945f7689635: Generated new TSK 0
I20260807 00:44:20.762478 32302 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:20.767493 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 32174
2026-08-07T00:44:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2220 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:20 dist-test-slave-h6n0 krb5kdc[32324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:20 dist-test-slave-h6n0 krb5kdc[32324](info): set up 2 sockets
Aug 07 00:44:20 dist-test-slave-h6n0 krb5kdc[32324](info): commencing operation
krb5kdc: starting...
W20260807 00:44:22.678363 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.868s user 0.000s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:22 dist-test-slave-h6n0 krb5kdc[32324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:22.813077 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36567
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36567
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:22.898784 32348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:22.898975 32348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:22.898998 32348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:22.900060 32348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:22.900100 32348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:22.900115 32348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:22.900127 32348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:22.900141 32348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:22.901564 32348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36567
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36567
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:22.901751 32348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:22.901923 32348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:22.907390 32360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:22.907433 32359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:22.907390 32363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:22.907609 32348 server_base.cc:1034] running on GCE node
I20260807 00:44:22.907917 32348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:22.908232 32348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:22.909428 32348 hybrid_clock.cc:648] HybridClock initialized: now 1786063462909409 us; error 35 us; skew 500 ppm
Aug 07 00:44:22 dist-test-slave-h6n0 krb5kdc[32324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:22.912199 32348 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:22.912636 32348 webserver.cc:460] Webserver started at http://127.0.0.1:43441/ using document root <none> and password file <none>
I20260807 00:44:22.912896 32348 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:22.912968 32348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:22.913123 32348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:22.914119 32348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47"
format_stamp: "Formatted at 2026-08-07 00:44:22 on dist-test-slave-h6n0"
server_key: "fa26e56ac0f7d2f785f87becf70cebe1"
server_key_iv: "ee676f824e4f03ed997d4bdde783f16f"
server_key_version: "encryptionkey@0"
I20260807 00:44:22.914484 32348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47"
format_stamp: "Formatted at 2026-08-07 00:44:22 on dist-test-slave-h6n0"
server_key: "fa26e56ac0f7d2f785f87becf70cebe1"
server_key_iv: "ee676f824e4f03ed997d4bdde783f16f"
server_key_version: "encryptionkey@0"
I20260807 00:44:22.915946 32348 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:22.916844 32374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:22.917114 32348 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:22.917240 32348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47"
format_stamp: "Formatted at 2026-08-07 00:44:22 on dist-test-slave-h6n0"
server_key: "fa26e56ac0f7d2f785f87becf70cebe1"
server_key_iv: "ee676f824e4f03ed997d4bdde783f16f"
server_key_version: "encryptionkey@0"
I20260807 00:44:22.917318 32348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:22.927183 32348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:22.929394 32348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:22.929557 32348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:22.934178 32348 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36567
I20260807 00:44:22.934260 32453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36567 every 8 connection(s)
I20260807 00:44:22.934636 32348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:22.935808 32455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:22.938421 32455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47: Bootstrap starting.
I20260807 00:44:22.939157 32455 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:22.939347 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 32348
I20260807 00:44:22.939459 32455 log.cc:824] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:22.939478 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:22.939726 1490 external_mini_cluster.cc:1428] Setting key d00ccf40eaddf8ddafd251c6dd26c1cb
I20260807 00:44:22.940364 32455 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47: No bootstrap required, opened a new log
I20260807 00:44:22.943593 32455 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
I20260807 00:44:22.943806 32455 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:22.943847 32455 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54ff7b47887c4a2ab2e8b61e4e2b5d47, State: Initialized, Role: FOLLOWER
I20260807 00:44:22.943994 32455 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
I20260807 00:44:22.944070 32455 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:22.944103 32455 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:22.944144 32455 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:22.944880 32455 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
I20260807 00:44:22.945017 32455 leader_election.cc:302] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54ff7b47887c4a2ab2e8b61e4e2b5d47; no voters:
I20260807 00:44:22.945228 32455 leader_election.cc:288] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:22.945557 32455 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:22.945385 32469 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:22.946938 32469 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [term 1 LEADER]: Becoming Leader. State: Replica: 54ff7b47887c4a2ab2e8b61e4e2b5d47, State: Running, Role: LEADER
I20260807 00:44:22.947120 32469 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } }
Aug 07 00:44:22 dist-test-slave-h6n0 krb5kdc[32324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063462, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:44:22.947719 32470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } } }
I20260807 00:44:22.947683 32480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54ff7b47887c4a2ab2e8b61e4e2b5d47. Latest consensus state: current_term: 1 leader_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54ff7b47887c4a2ab2e8b61e4e2b5d47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36567 } } }
I20260807 00:44:22.948011 32480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:22.947798 32470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47 [sys.catalog]: This master's current role is: LEADER
W20260807 00:44:22.948218 32487 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:22.948271 32487 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:22.948623 32491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:22.948837 32491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:22.950805 32491 catalog_manager.cc:1347] Generated new cluster ID: c23b08b3c94f479fbc562d9d429de4b1
I20260807 00:44:22.950870 32491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:22.952445 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:22.940258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:36885 (local address 127.0.0.1:36567)
0807 00:44:22.940452 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:22.940479 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:22.940515 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:44:22.941102 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:22.941109 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:22.941161 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:22.941275 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:22.942196 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:22.943123 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:22.944309 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:22.944722 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:22.944813 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:22.948168 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:22.948172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:22.948172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:22.948185 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:22.950488 (+ 2303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:22.951065 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:22.951069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:22.951069 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:22.951198 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:22.951479 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:22.951481 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:22.951481 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:22.951785 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:22.951961 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:22.952097 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 00:44:22.952215 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
I20260807 00:44:22.953629 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:22.960472 32491 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:22.961094 32491 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:22.963635 32465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:22.955283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:40729 (local address 127.0.0.1:36567)
0807 00:44:22.955324 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:22.955326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:22.955641 (+ 315us) server_negotiation.cc:407] Connection header received
0807 00:44:22.955736 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:22.955737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:22.955746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:22.955777 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:22.956528 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:22.957265 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:22.958301 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:22.958669 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:22.958714 (+ 45us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:22.960238 (+ 1524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:22.960240 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:22.960240 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:22.960247 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:22.961995 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:22.962424 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:22.962426 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:22.962426 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:22.962478 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:22.962763 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:22.962764 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:22.962764 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:22.963071 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:22.963186 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:22.963456 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 00:44:22.963527 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:44:22.980436 32491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54ff7b47887c4a2ab2e8b61e4e2b5d47: Generated new TSK 0
I20260807 00:44:22.980680 32491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:22.986354 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 32348
W20260807 00:44:23.005542 32319 connection.cc:535] client connection to 127.0.0.1:36567 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36567): Connection reset by peer (error 104)
2026-08-07T00:44:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2210 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:23 dist-test-slave-h6n0 krb5kdc[32507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:23 dist-test-slave-h6n0 krb5kdc[32507](info): set up 2 sockets
Aug 07 00:44:23 dist-test-slave-h6n0 krb5kdc[32507](info): commencing operation
krb5kdc: starting...
W20260807 00:44:24.236488 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.217s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:24 dist-test-slave-h6n0 krb5kdc[32507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:24.368713 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:24.453610 32534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:24.453841 32534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:24.453868 32534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:24.455037 32534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:24.455083 32534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:24.455098 32534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:24.455111 32534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:24.455123 32534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:24.456507 32534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:24.456689 32534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:24.456846 32534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:24.461850 32539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:24.461817 32544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:24.462013 32540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:24.462066 32534 server_base.cc:1034] running on GCE node
I20260807 00:44:24.462441 32534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:24.462872 32534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:24.464043 32534 hybrid_clock.cc:648] HybridClock initialized: now 1786063464464031 us; error 48 us; skew 500 ppm
Aug 07 00:44:24 dist-test-slave-h6n0 krb5kdc[32507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063464, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:24.467031 32534 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:24.467569 32534 webserver.cc:460] Webserver started at http://127.0.0.1:38773/ using document root <none> and password file <none>
I20260807 00:44:24.467832 32534 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:24.467885 32534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:24.468082 32534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:24.469274 32534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8"
format_stamp: "Formatted at 2026-08-07 00:44:24 on dist-test-slave-h6n0"
server_key: "8c60f9912e71d09480afd550270848af"
server_key_iv: "5303a769972f000a6cdfbf637c9f68c4"
server_key_version: "encryptionkey@0"
I20260807 00:44:24.469678 32534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8"
format_stamp: "Formatted at 2026-08-07 00:44:24 on dist-test-slave-h6n0"
server_key: "8c60f9912e71d09480afd550270848af"
server_key_iv: "5303a769972f000a6cdfbf637c9f68c4"
server_key_version: "encryptionkey@0"
I20260807 00:44:24.471302 32534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:24.472205 32555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:24.472460 32534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:24.472586 32534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8"
format_stamp: "Formatted at 2026-08-07 00:44:24 on dist-test-slave-h6n0"
server_key: "8c60f9912e71d09480afd550270848af"
server_key_iv: "5303a769972f000a6cdfbf637c9f68c4"
server_key_version: "encryptionkey@0"
I20260807 00:44:24.472675 32534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:24.489506 32534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:24.492146 32534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:24.492295 32534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:24.497367 32534 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46231
I20260807 00:44:24.497452 32642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46231 every 8 connection(s)
I20260807 00:44:24.497830 32534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:24.498966 32644 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:24.502202 32644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8: Bootstrap starting.
I20260807 00:44:24.502980 32644 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:24.503286 32644 log.cc:824] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:24.504094 32644 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8: No bootstrap required, opened a new log
I20260807 00:44:24.504506 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 32534
I20260807 00:44:24.504658 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:24.504920 1490 external_mini_cluster.cc:1428] Setting key a64ad3bb045bfabeaa85ff7a0d226285
I20260807 00:44:24.506222 32644 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260807 00:44:24.506392 32644 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:24.506460 32644 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63cdc585e47b4a3ea31abdd2e96c8fe8, State: Initialized, Role: FOLLOWER
I20260807 00:44:24.506651 32644 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260807 00:44:24.506753 32644 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:24.506796 32644 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:24.506850 32644 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:24.507458 32644 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260807 00:44:24.507581 32644 leader_election.cc:302] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63cdc585e47b4a3ea31abdd2e96c8fe8; no voters:
I20260807 00:44:24.507776 32644 leader_election.cc:288] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:24.507805 32652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:24.507932 32652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [term 1 LEADER]: Becoming Leader. State: Replica: 63cdc585e47b4a3ea31abdd2e96c8fe8, State: Running, Role: LEADER
I20260807 00:44:24.508107 32644 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:24.508140 32652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } }
I20260807 00:44:24.508711 32654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } } }
I20260807 00:44:24.508841 32654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:24.508919 32655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63cdc585e47b4a3ea31abdd2e96c8fe8. Latest consensus state: current_term: 1 leader_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63cdc585e47b4a3ea31abdd2e96c8fe8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46231 } } }
I20260807 00:44:24.509400 32655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:24.510100 32669 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:24.510846 32669 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:24.512276 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:24.505558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:57161 (local address 127.0.0.1:46231)
0807 00:44:24.505784 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:24.505794 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:24.505824 (+ 30us) server_negotiation.cc:407] Connection header received
0807 00:44:24.506221 (+ 397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:24.506229 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:24.506292 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:24.506464 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:24.506746 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:24.508406 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:24.509562 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:24.510320 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:24.510438 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:24.511292 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:24.511297 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:24.511298 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:24.511619 (+ 321us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:24.511662 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:24.511676 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:24.511782 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:24.511963 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 00:44:24.512000 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260807 00:44:24.513039 32669 catalog_manager.cc:1347] Generated new cluster ID: 49742f1dfebb4ada97146488fe0c9884
I20260807 00:44:24.513095 32669 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:24.513433 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:24.517940 32651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:24.514943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:35211 (local address 127.0.0.1:46231)
0807 00:44:24.514981 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:24.514983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:24.515119 (+ 136us) server_negotiation.cc:407] Connection header received
0807 00:44:24.515192 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:24.515194 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:24.515208 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:24.515267 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:24.515423 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:24.515890 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:24.516644 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:24.516896 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:24.516974 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:24.517684 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:24.517686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:24.517686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:24.517690 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:24.517705 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:24.517710 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:24.517765 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:24.517827 (+ 62us) server_negotiation.cc:299] Negotiation successful
0807 00:44:24.517839 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 00:44:24.525189 32669 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:24.525942 32669 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:24.529409 32669 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63cdc585e47b4a3ea31abdd2e96c8fe8: Generated new TSK 0
I20260807 00:44:24.529609 32669 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:24.543166 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 32534
W20260807 00:44:24.559517 32502 connection.cc:535] client connection to 127.0.0.1:46231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46231): Connection reset by peer (error 104)
2026-08-07T00:44:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:24 dist-test-slave-h6n0 krb5kdc[32689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:24 dist-test-slave-h6n0 krb5kdc[32689](info): set up 2 sockets
Aug 07 00:44:24 dist-test-slave-h6n0 krb5kdc[32689](info): commencing operation
krb5kdc: starting...
W20260807 00:44:25.800127 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:25 dist-test-slave-h6n0 krb5kdc[32689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:44:25.938839 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37269
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37269
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:26.030224 32707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:26.030490 32707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:26.030529 32707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:26.031698 32707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:26.031769 32707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:26.031795 32707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:26.031824 32707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:26.031855 32707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:26.033413 32707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37269
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:26.033716 32707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:26.033967 32707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:26.039222 32717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:26.039227 32725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:26.039225 32721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:26.039634 32707 server_base.cc:1034] running on GCE node
I20260807 00:44:26.039901 32707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:26.040282 32707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:26.041462 32707 hybrid_clock.cc:648] HybridClock initialized: now 1786063466041437 us; error 46 us; skew 500 ppm
Aug 07 00:44:26 dist-test-slave-h6n0 krb5kdc[32689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063466, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:26.050195 32707 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:44:26.050827 32707 webserver.cc:460] Webserver started at http://127.0.0.1:45377/ using document root <none> and password file <none>
I20260807 00:44:26.051090 32707 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:26.051172 32707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:26.051335 32707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:26.052523 32707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "b32bcfc8fa84416d9ee098d8eaf8a799"
format_stamp: "Formatted at 2026-08-07 00:44:26 on dist-test-slave-h6n0"
server_key: "7a581176b38bed3f3872ea324fea1cc9"
server_key_iv: "2d17700853632bc2d16925ddac757608"
server_key_version: "encryptionkey@0"
I20260807 00:44:26.052924 32707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "b32bcfc8fa84416d9ee098d8eaf8a799"
format_stamp: "Formatted at 2026-08-07 00:44:26 on dist-test-slave-h6n0"
server_key: "7a581176b38bed3f3872ea324fea1cc9"
server_key_iv: "2d17700853632bc2d16925ddac757608"
server_key_version: "encryptionkey@0"
I20260807 00:44:26.054418 32707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:26.055321 32742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:26.055578 32707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:26.055699 32707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "b32bcfc8fa84416d9ee098d8eaf8a799"
format_stamp: "Formatted at 2026-08-07 00:44:26 on dist-test-slave-h6n0"
server_key: "7a581176b38bed3f3872ea324fea1cc9"
server_key_iv: "2d17700853632bc2d16925ddac757608"
server_key_version: "encryptionkey@0"
I20260807 00:44:26.055778 32707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:26.059449 32707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:26.061806 32707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:26.061970 32707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:26.067415 32707 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37269
I20260807 00:44:26.067487 348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37269 every 8 connection(s)
I20260807 00:44:26.067848 32707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:26.068872 349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:26.071425 349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799: Bootstrap starting.
I20260807 00:44:26.072057 349 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:26.072373 349 log.cc:824] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:26.073189 349 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799: No bootstrap required, opened a new log
I20260807 00:44:26.074720 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 32707
I20260807 00:44:26.074870 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:26.075088 1490 external_mini_cluster.cc:1428] Setting key 50723b5c99a1c7151258c01865c036e3
I20260807 00:44:26.075158 349 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37269 } }
I20260807 00:44:26.075335 349 raft_consensus.cc:374] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:26.075361 349 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32bcfc8fa84416d9ee098d8eaf8a799, State: Initialized, Role: FOLLOWER
I20260807 00:44:26.075562 349 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37269 } }
I20260807 00:44:26.075654 349 raft_consensus.cc:388] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:26.075690 349 raft_consensus.cc:482] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:26.075810 349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:26.076345 349 raft_consensus.cc:504] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37269 } }
I20260807 00:44:26.076478 349 leader_election.cc:302] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b32bcfc8fa84416d9ee098d8eaf8a799; no voters:
I20260807 00:44:26.076676 349 leader_election.cc:288] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:26.076716 356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:26.076987 356 raft_consensus.cc:686] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [term 1 LEADER]: Becoming Leader. State: Replica: b32bcfc8fa84416d9ee098d8eaf8a799, State: Running, Role: LEADER
I20260807 00:44:26.077032 349 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:26.077195 356 consensus_queue.cc:237] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37269 } }
I20260807 00:44:26.078104 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:26.075654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:54605 (local address 127.0.0.1:37269)
0807 00:44:26.075889 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:26.075901 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:26.075942 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:44:26.076370 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:26.076377 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:26.076406 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:26.076581 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:26.077399 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:26.077403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:26.077404 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:26.077408 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:26.077458 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:26.077473 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:26.077554 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:26.077754 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 00:44:26.077817 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":104,"threads_started":1}
I20260807 00:44:26.078677 360 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b32bcfc8fa84416d9ee098d8eaf8a799. Latest consensus state: current_term: 1 leader_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37269 } } }
I20260807 00:44:26.078626 357 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b32bcfc8fa84416d9ee098d8eaf8a799" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37269 } } }
I20260807 00:44:26.078799 360 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:26.078840 357 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:26.079159 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:26.079205 373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:26.079766 373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:26.081943 373 catalog_manager.cc:1347] Generated new cluster ID: cfb7d5f352234d9f9a009cfea674cce3
I20260807 00:44:26.082003 373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:26.082477 355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:26.081116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.52:54453 (local address 127.0.0.1:37269)
0807 00:44:26.081193 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:26.081196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:26.081211 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:44:26.081238 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:26.081239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:26.081246 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:26.081316 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:26.082098 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:26.082100 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:26.082100 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:26.082108 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:26.082182 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:26.082196 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:26.082242 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:26.082394 (+ 152us) server_negotiation.cc:299] Negotiation successful
0807 00:44:26.082413 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:44:26.090332 373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:26.090516 373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:26.103909 373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b32bcfc8fa84416d9ee098d8eaf8a799: Generated new TSK 0
I20260807 00:44:26.104102 373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:26.110365 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 32707
2026-08-07T00:44:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1565 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:26 dist-test-slave-h6n0 krb5kdc[392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:26 dist-test-slave-h6n0 krb5kdc[392](info): set up 2 sockets
Aug 07 00:44:26 dist-test-slave-h6n0 krb5kdc[392](info): commencing operation
krb5kdc: starting...
W20260807 00:44:27.395989 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:27 dist-test-slave-h6n0 krb5kdc[392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:27.528901 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:34355
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44351
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:34355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:27.614212 411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:27.614408 411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:27.614432 411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:27.615571 411 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:27.615619 411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:27.615634 411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:27.615648 411 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:27.615660 411 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:27.617053 411 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44351
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:34355
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:34355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:27.617237 411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:27.617393 411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:27.622591 423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:27.622606 432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:27.622599 422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:27.622874 411 server_base.cc:1034] running on GCE node
I20260807 00:44:27.623227 411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:27.623593 411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:27.624804 411 hybrid_clock.cc:648] HybridClock initialized: now 1786063467624779 us; error 56 us; skew 500 ppm
Aug 07 00:44:27 dist-test-slave-h6n0 krb5kdc[392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:27.627815 411 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:27.628273 411 webserver.cc:460] Webserver started at http://127.1.116.190:46805/ using document root <none> and password file <none>
I20260807 00:44:27.628525 411 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:27.628572 411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:27.628736 411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:27.629900 411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "be6ab16ed8dd46a4be4e64b032263955"
format_stamp: "Formatted at 2026-08-07 00:44:27 on dist-test-slave-h6n0"
server_key: "c0b87805a29b0d20f3272de36d5cd508"
server_key_iv: "245bb74c323b6436f5db57611fcbe4ad"
server_key_version: "encryptionkey@0"
I20260807 00:44:27.630405 411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "be6ab16ed8dd46a4be4e64b032263955"
format_stamp: "Formatted at 2026-08-07 00:44:27 on dist-test-slave-h6n0"
server_key: "c0b87805a29b0d20f3272de36d5cd508"
server_key_iv: "245bb74c323b6436f5db57611fcbe4ad"
server_key_version: "encryptionkey@0"
I20260807 00:44:27.632418 411 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260807 00:44:27.633431 445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:27.633692 411 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:27.633821 411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "be6ab16ed8dd46a4be4e64b032263955"
format_stamp: "Formatted at 2026-08-07 00:44:27 on dist-test-slave-h6n0"
server_key: "c0b87805a29b0d20f3272de36d5cd508"
server_key_iv: "245bb74c323b6436f5db57611fcbe4ad"
server_key_version: "encryptionkey@0"
I20260807 00:44:27.633925 411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:27.648427 411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:27.651190 411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:27.651388 411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:27.656595 411 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:34355
I20260807 00:44:27.656646 524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:34355 every 8 connection(s)
I20260807 00:44:27.657105 411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:27.658048 525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:27.660400 525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955: Bootstrap starting.
I20260807 00:44:27.661017 525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:27.661319 525 log.cc:824] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:27.662177 525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955: No bootstrap required, opened a new log
I20260807 00:44:27.663815 525 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ab16ed8dd46a4be4e64b032263955" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 34355 } }
I20260807 00:44:27.663954 525 raft_consensus.cc:374] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:27.663978 525 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6ab16ed8dd46a4be4e64b032263955, State: Initialized, Role: FOLLOWER
I20260807 00:44:27.664150 525 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ab16ed8dd46a4be4e64b032263955" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 34355 } }
I20260807 00:44:27.664216 525 raft_consensus.cc:388] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:27.664238 525 raft_consensus.cc:482] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:27.664306 525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:27.664670 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 411
I20260807 00:44:27.664810 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:27.665076 1490 external_mini_cluster.cc:1428] Setting key ea92522f88b1270ad90d07c94776ff22
I20260807 00:44:27.665498 525 raft_consensus.cc:504] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ab16ed8dd46a4be4e64b032263955" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 34355 } }
I20260807 00:44:27.665611 525 leader_election.cc:302] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be6ab16ed8dd46a4be4e64b032263955; no voters:
I20260807 00:44:27.665833 525 leader_election.cc:288] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:27.665887 530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:27.666090 530 raft_consensus.cc:686] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [term 1 LEADER]: Becoming Leader. State: Replica: be6ab16ed8dd46a4be4e64b032263955, State: Running, Role: LEADER
I20260807 00:44:27.666180 525 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:27.666280 530 consensus_queue.cc:237] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ab16ed8dd46a4be4e64b032263955" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 34355 } }
I20260807 00:44:27.666775 534 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [sys.catalog]: SysCatalogTable state changed. Reason: New leader be6ab16ed8dd46a4be4e64b032263955. Latest consensus state: current_term: 1 leader_uuid: "be6ab16ed8dd46a4be4e64b032263955" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ab16ed8dd46a4be4e64b032263955" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 34355 } } }
I20260807 00:44:27.666905 534 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:27.667539 531 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be6ab16ed8dd46a4be4e64b032263955" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ab16ed8dd46a4be4e64b032263955" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 34355 } } }
I20260807 00:44:27.667737 531 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:27.667177 547 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:27.668534 547 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:27.670267 547 catalog_manager.cc:1347] Generated new cluster ID: 696c7d889b394d80b8119b3349e92bcb
I20260807 00:44:27.670327 547 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 07 00:44:27 dist-test-slave-h6n0 krb5kdc[392](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063467, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:27.677917 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:27.665657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54485 (local address 127.1.116.190:34355)
0807 00:44:27.665858 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:27.665870 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:27.665940 (+ 70us) server_negotiation.cc:407] Connection header received
0807 00:44:27.666095 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:27.666104 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:27.666176 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:27.666354 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:27.667286 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:27.668891 (+ 1605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:27.669977 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:27.670632 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:27.670738 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:27.673774 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:27.673779 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:27.673779 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:27.673793 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:27.676033 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:27.676540 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:27.676543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:27.676544 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:27.676671 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:27.676941 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:27.676942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:27.676943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:27.677271 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:27.677451 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:27.677590 (+ 139us) server_negotiation.cc:299] Negotiation successful
0807 00:44:27.677702 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":109,"threads_started":1}
I20260807 00:44:27.679365 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:27.681483 547 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:27.682022 547 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:27.686285 547 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be6ab16ed8dd46a4be4e64b032263955: Generated new TSK 0
I20260807 00:44:27.686474 547 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:27.688956 529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:27.680938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40179 (local address 127.1.116.190:34355)
0807 00:44:27.681165 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:27.681167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:27.681197 (+ 30us) server_negotiation.cc:407] Connection header received
0807 00:44:27.681412 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:27.681413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:27.681427 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:27.681498 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:27.682206 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:27.683072 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:27.684096 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:27.684428 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:27.684486 (+ 58us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:27.685929 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:27.685931 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:27.685932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:27.685940 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:27.687675 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:27.688178 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:27.688179 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:27.688180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:27.688235 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:27.688486 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:27.688486 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:27.688486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:27.688588 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:27.688674 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:27.688842 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:44:27.688894 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":10,"server-negotiator.queue_time_us":200}
I20260807 00:44:27.691280 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 411
W20260807 00:44:27.705152 388 connection.cc:535] client connection to 127.1.116.190:34355 recv error: Network error: failed to read from TLS socket (remote: 127.1.116.190:34355): Connection reset by peer (error 104)
2026-08-07T00:44:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1580 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:27 dist-test-slave-h6n0 krb5kdc[571](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:27 dist-test-slave-h6n0 krb5kdc[571](info): set up 2 sockets
Aug 07 00:44:27 dist-test-slave-h6n0 krb5kdc[571](info): commencing operation
krb5kdc: starting...
W20260807 00:44:28.951872 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:29 dist-test-slave-h6n0 krb5kdc[571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063469, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:29.082902 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:37783
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:39929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:37783
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:29.165643 593 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:29.165889 593 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:29.165931 593 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:29.167140 593 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:29.167207 593 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:29.167232 593 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:29.167259 593 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:29.167291 593 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:29.168792 593 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:39929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:37783
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:37783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.593
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:29.169080 593 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:29.169308 593 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:29.174129 606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:29.174196 601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:29.174209 604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:29.174747 593 server_base.cc:1034] running on GCE node
I20260807 00:44:29.175136 593 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:29.175515 593 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:29.176700 593 hybrid_clock.cc:648] HybridClock initialized: now 1786063469176690 us; error 45 us; skew 500 ppm
Aug 07 00:44:29 dist-test-slave-h6n0 krb5kdc[571](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063469, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:29.179939 593 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:29.180459 593 webserver.cc:460] Webserver started at http://127.1.116.190:44317/ using document root <none> and password file <none>
I20260807 00:44:29.180739 593 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:29.180809 593 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:29.180969 593 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:29.182176 593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "cb4292863ca043a98edf849b3edc53e8"
format_stamp: "Formatted at 2026-08-07 00:44:29 on dist-test-slave-h6n0"
server_key: "61a5b6042d517ce2c676be008e82e7d7"
server_key_iv: "4f1020dd3b7151ec50c9841739513a92"
server_key_version: "encryptionkey@0"
I20260807 00:44:29.182710 593 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "cb4292863ca043a98edf849b3edc53e8"
format_stamp: "Formatted at 2026-08-07 00:44:29 on dist-test-slave-h6n0"
server_key: "61a5b6042d517ce2c676be008e82e7d7"
server_key_iv: "4f1020dd3b7151ec50c9841739513a92"
server_key_version: "encryptionkey@0"
I20260807 00:44:29.184407 593 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260807 00:44:29.185213 618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:29.185468 593 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:29.185546 593 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "cb4292863ca043a98edf849b3edc53e8"
format_stamp: "Formatted at 2026-08-07 00:44:29 on dist-test-slave-h6n0"
server_key: "61a5b6042d517ce2c676be008e82e7d7"
server_key_iv: "4f1020dd3b7151ec50c9841739513a92"
server_key_version: "encryptionkey@0"
I20260807 00:44:29.185652 593 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:29.212821 593 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:29.215229 593 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:29.215397 593 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:29.220698 593 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:37783
I20260807 00:44:29.220804 697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:37783 every 8 connection(s)
I20260807 00:44:29.221233 593 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:29.222263 698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:29.224701 698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8: Bootstrap starting.
I20260807 00:44:29.225342 698 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:29.225646 698 log.cc:824] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:29.226507 698 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8: No bootstrap required, opened a new log
I20260807 00:44:29.228204 698 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4292863ca043a98edf849b3edc53e8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37783 } }
I20260807 00:44:29.228348 698 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:29.228370 698 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb4292863ca043a98edf849b3edc53e8, State: Initialized, Role: FOLLOWER
I20260807 00:44:29.228533 698 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4292863ca043a98edf849b3edc53e8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37783 } }
I20260807 00:44:29.228605 698 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:29.228626 698 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:29.228696 698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:29.229259 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 593
I20260807 00:44:29.229413 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:29.229658 1490 external_mini_cluster.cc:1428] Setting key 4b8f9c2e077b56c8ec5c942aa4a8cdfd
I20260807 00:44:29.229696 698 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4292863ca043a98edf849b3edc53e8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37783 } }
I20260807 00:44:29.229851 698 leader_election.cc:302] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb4292863ca043a98edf849b3edc53e8; no voters:
I20260807 00:44:29.230049 698 leader_election.cc:288] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:29.230127 705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:29.230369 705 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [term 1 LEADER]: Becoming Leader. State: Replica: cb4292863ca043a98edf849b3edc53e8, State: Running, Role: LEADER
I20260807 00:44:29.230432 698 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:29.231140 705 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4292863ca043a98edf849b3edc53e8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37783 } }
I20260807 00:44:29.231926 708 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb4292863ca043a98edf849b3edc53e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4292863ca043a98edf849b3edc53e8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37783 } } }
I20260807 00:44:29.232064 709 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb4292863ca043a98edf849b3edc53e8. Latest consensus state: current_term: 1 leader_uuid: "cb4292863ca043a98edf849b3edc53e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb4292863ca043a98edf849b3edc53e8" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37783 } } }
I20260807 00:44:29.232095 708 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:29.232123 709 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:29.232331 730 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:29.233110 730 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:29.235003 730 catalog_manager.cc:1347] Generated new cluster ID: d5cc60c8ae7d4d8690ad547e727ae769
I20260807 00:44:29.235071 730 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:29.236471 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:29.230266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34863 (local address 127.1.116.190:37783)
0807 00:44:29.230621 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:29.230632 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:29.230667 (+ 35us) server_negotiation.cc:407] Connection header received
0807 00:44:29.230786 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:29.230791 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:29.230834 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:29.230979 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:29.231216 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:29.233150 (+ 1934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:29.234172 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:29.234781 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:29.234912 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:29.235840 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:29.235843 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:29.235844 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:29.236028 (+ 184us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:29.236061 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:29.236071 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:29.236163 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:29.236244 (+ 81us) server_negotiation.cc:299] Negotiation successful
0807 00:44:29.236273 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":87,"threads_started":1}
I20260807 00:44:29.237634 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:29.242760 711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:29.239570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45149 (local address 127.1.116.190:37783)
0807 00:44:29.239612 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:29.239614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:29.239622 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:44:29.239679 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:29.239680 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:29.239690 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:29.239733 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:29.239924 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:29.240709 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:29.241429 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:29.241658 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:29.241730 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:29.242498 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:29.242499 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:29.242499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:29.242503 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:29.242517 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:29.242521 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:29.242604 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:29.242667 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 00:44:29.242681 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 00:44:29.255311 730 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:29.255918 730 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:29.262451 730 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb4292863ca043a98edf849b3edc53e8: Generated new TSK 0
I20260807 00:44:29.262661 730 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:29.269379 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 593
W20260807 00:44:29.284533 566 connection.cc:535] client connection to 127.1.116.190:37783 recv error: Network error: failed to read from TLS socket (remote: 127.1.116.190:37783): Connection reset by peer (error 104)
2026-08-07T00:44:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1579 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:29 dist-test-slave-h6n0 krb5kdc[746](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:29 dist-test-slave-h6n0 krb5kdc[746](info): set up 2 sockets
Aug 07 00:44:29 dist-test-slave-h6n0 krb5kdc[746](info): commencing operation
krb5kdc: starting...
W20260807 00:44:30.533790 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:30 dist-test-slave-h6n0 krb5kdc[746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:30.669502 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43251
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:33591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:43251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:30.752288 769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:30.752490 769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:30.752513 769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:30.753652 769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:30.753697 769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:30.753712 769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:30.753726 769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:30.753738 769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:30.755453 769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:33591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:43251
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:30.755714 769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:30.755970 769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:30.761363 777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:30.761394 774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:30.761410 775 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:30.761806 769 server_base.cc:1034] running on GCE node
I20260807 00:44:30.762115 769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:30.762571 769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:30.763784 769 hybrid_clock.cc:648] HybridClock initialized: now 1786063470763761 us; error 44 us; skew 500 ppm
Aug 07 00:44:30 dist-test-slave-h6n0 krb5kdc[746](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063470, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:30.766893 769 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:30.767432 769 webserver.cc:460] Webserver started at http://127.1.116.190:41349/ using document root <none> and password file <none>
I20260807 00:44:30.767731 769 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:30.767804 769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:30.767969 769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:30.769197 769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "f21846d91e804039a67bd7d205dcd02c"
format_stamp: "Formatted at 2026-08-07 00:44:30 on dist-test-slave-h6n0"
server_key: "fa0fc2de8596cc7d1e5e9fb312d862a7"
server_key_iv: "a27c664071c5ad7589add52f87819d0b"
server_key_version: "encryptionkey@0"
I20260807 00:44:30.769625 769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "f21846d91e804039a67bd7d205dcd02c"
format_stamp: "Formatted at 2026-08-07 00:44:30 on dist-test-slave-h6n0"
server_key: "fa0fc2de8596cc7d1e5e9fb312d862a7"
server_key_iv: "a27c664071c5ad7589add52f87819d0b"
server_key_version: "encryptionkey@0"
I20260807 00:44:30.771260 769 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:30.772189 789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:30.772439 769 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:30.772547 769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "f21846d91e804039a67bd7d205dcd02c"
format_stamp: "Formatted at 2026-08-07 00:44:30 on dist-test-slave-h6n0"
server_key: "fa0fc2de8596cc7d1e5e9fb312d862a7"
server_key_iv: "a27c664071c5ad7589add52f87819d0b"
server_key_version: "encryptionkey@0"
I20260807 00:44:30.772655 769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:30.776465 769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:30.778865 769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:30.779052 769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:30.784132 769 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:43251
I20260807 00:44:30.784194 875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:43251 every 8 connection(s)
I20260807 00:44:30.784574 769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:30.785748 876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:30.785827 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 769
I20260807 00:44:30.785961 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:30.786213 1490 external_mini_cluster.cc:1428] Setting key d025e8f4afbce6573474b59938f2488d
I20260807 00:44:30.789574 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:30.786971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53453 (local address 127.1.116.190:43251)
0807 00:44:30.787283 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:30.787294 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:30.787331 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:44:30.787871 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:30.787880 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:30.787916 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:30.788106 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:30.789059 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:30.789063 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:30.789064 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:30.789069 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:30.789118 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:30.789132 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:30.789219 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:30.789345 (+ 126us) server_negotiation.cc:299] Negotiation successful
0807 00:44:30.789389 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":162,"threads_started":1}
I20260807 00:44:30.789880 876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c: Bootstrap starting.
I20260807 00:44:30.790652 876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:30.791014 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:30.791028 876 log.cc:824] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:30.791926 876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c: No bootstrap required, opened a new log
I20260807 00:44:30.794365 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:30.792876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51165 (local address 127.1.116.190:43251)
0807 00:44:30.792951 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:30.792954 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:30.793161 (+ 207us) server_negotiation.cc:407] Connection header received
0807 00:44:30.793192 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:30.793192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:30.793213 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:30.793273 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:30.793951 (+ 678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:30.793952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:30.793952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:30.793956 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:30.793980 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:30.793984 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:30.794049 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:30.794252 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 00:44:30.794270 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 00:44:30.794625 876 raft_consensus.cc:348] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21846d91e804039a67bd7d205dcd02c" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43251 } }
I20260807 00:44:30.794800 876 raft_consensus.cc:374] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:30.794837 876 raft_consensus.cc:729] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f21846d91e804039a67bd7d205dcd02c, State: Initialized, Role: FOLLOWER
I20260807 00:44:30.794962 876 consensus_queue.cc:260] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21846d91e804039a67bd7d205dcd02c" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43251 } }
I20260807 00:44:30.795070 876 raft_consensus.cc:388] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:30.795130 876 raft_consensus.cc:482] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:30.795184 876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:30.795715 876 raft_consensus.cc:504] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21846d91e804039a67bd7d205dcd02c" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43251 } }
I20260807 00:44:30.795804 876 leader_election.cc:302] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f21846d91e804039a67bd7d205dcd02c; no voters:
I20260807 00:44:30.795979 876 leader_election.cc:288] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:30.796106 890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:30.796262 876 sys_catalog.cc:564] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:30.796363 890 raft_consensus.cc:686] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [term 1 LEADER]: Becoming Leader. State: Replica: f21846d91e804039a67bd7d205dcd02c, State: Running, Role: LEADER
I20260807 00:44:30.796486 890 consensus_queue.cc:237] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21846d91e804039a67bd7d205dcd02c" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43251 } }
I20260807 00:44:30.797338 893 sys_catalog.cc:455] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [sys.catalog]: SysCatalogTable state changed. Reason: New leader f21846d91e804039a67bd7d205dcd02c. Latest consensus state: current_term: 1 leader_uuid: "f21846d91e804039a67bd7d205dcd02c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21846d91e804039a67bd7d205dcd02c" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43251 } } }
I20260807 00:44:30.797343 891 sys_catalog.cc:455] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f21846d91e804039a67bd7d205dcd02c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f21846d91e804039a67bd7d205dcd02c" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43251 } } }
I20260807 00:44:30.797477 893 sys_catalog.cc:458] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:30.797489 891 sys_catalog.cc:458] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:30.797771 905 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:30.798516 905 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:30.800549 905 catalog_manager.cc:1347] Generated new cluster ID: d11e3fa909eb403d8c269ec34bab7458
I20260807 00:44:30.800622 905 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:30.836337 905 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:30.836583 905 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:30.846437 905 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f21846d91e804039a67bd7d205dcd02c: Generated new TSK 0
I20260807 00:44:30.846695 905 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:30.856815 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 769
2026-08-07T00:44:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:30 dist-test-slave-h6n0 krb5kdc[921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:30 dist-test-slave-h6n0 krb5kdc[921](info): set up 2 sockets
Aug 07 00:44:30 dist-test-slave-h6n0 krb5kdc[921](info): commencing operation
krb5kdc: starting...
W20260807 00:44:32.111948 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:32 dist-test-slave-h6n0 krb5kdc[921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:32.247079 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:37143
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:41609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:37143
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:32.333513 942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:32.333740 942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:32.333786 942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:32.335000 942 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:32.335069 942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:32.335098 942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:32.335127 942 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:32.335161 942 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:32.336664 942 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:41609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:37143
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:37143
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:32.336941 942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:32.337198 942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:32.342377 950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:32.342448 952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:32.342435 956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:32.342692 942 server_base.cc:1034] running on GCE node
I20260807 00:44:32.343222 942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:32.343581 942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:32.344751 942 hybrid_clock.cc:648] HybridClock initialized: now 1786063472344721 us; error 50 us; skew 500 ppm
Aug 07 00:44:32 dist-test-slave-h6n0 krb5kdc[921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063472, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:32.347424 942 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:32.347878 942 webserver.cc:460] Webserver started at http://127.1.116.190:44747/ using document root <none> and password file <none>
I20260807 00:44:32.348140 942 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:32.348208 942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:32.348358 942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:32.349371 942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "b7238ca808804dddaf39389d335f4eae"
format_stamp: "Formatted at 2026-08-07 00:44:32 on dist-test-slave-h6n0"
server_key: "959e750f46ce9acd88236bb6e4cc16c2"
server_key_iv: "8daa74eec4489c7670253578d870ad05"
server_key_version: "encryptionkey@0"
I20260807 00:44:32.349829 942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "b7238ca808804dddaf39389d335f4eae"
format_stamp: "Formatted at 2026-08-07 00:44:32 on dist-test-slave-h6n0"
server_key: "959e750f46ce9acd88236bb6e4cc16c2"
server_key_iv: "8daa74eec4489c7670253578d870ad05"
server_key_version: "encryptionkey@0"
I20260807 00:44:32.351863 942 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260807 00:44:32.352777 965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:32.352952 942 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:32.353053 942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "b7238ca808804dddaf39389d335f4eae"
format_stamp: "Formatted at 2026-08-07 00:44:32 on dist-test-slave-h6n0"
server_key: "959e750f46ce9acd88236bb6e4cc16c2"
server_key_iv: "8daa74eec4489c7670253578d870ad05"
server_key_version: "encryptionkey@0"
I20260807 00:44:32.353116 942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:32.391940 942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:32.394357 942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:32.394477 942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:32.399128 942 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:37143
I20260807 00:44:32.399279 1047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:37143 every 8 connection(s)
I20260807 00:44:32.399572 942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:32.400707 1048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:32.403728 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 942
I20260807 00:44:32.403769 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae: Bootstrap starting.
I20260807 00:44:32.403887 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:32.404156 1490 external_mini_cluster.cc:1428] Setting key bfb45f256ce4b0e7a209419ccee63ce8
I20260807 00:44:32.404683 1048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:32.405140 1048 log.cc:824] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:32.406085 1048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae: No bootstrap required, opened a new log
I20260807 00:44:32.409049 1048 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7238ca808804dddaf39389d335f4eae" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37143 } }
I20260807 00:44:32.409269 1048 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:32.409430 1048 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7238ca808804dddaf39389d335f4eae, State: Initialized, Role: FOLLOWER
I20260807 00:44:32.409621 1048 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7238ca808804dddaf39389d335f4eae" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37143 } }
I20260807 00:44:32.409754 1048 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:32.409808 1048 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:32.409868 1048 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:32.410897 1048 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7238ca808804dddaf39389d335f4eae" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37143 } }
I20260807 00:44:32.411038 1048 leader_election.cc:302] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7238ca808804dddaf39389d335f4eae; no voters:
I20260807 00:44:32.411258 1048 leader_election.cc:288] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:32.411360 1055 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:32.411588 1055 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [term 1 LEADER]: Becoming Leader. State: Replica: b7238ca808804dddaf39389d335f4eae, State: Running, Role: LEADER
I20260807 00:44:32.411676 1048 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:32.411798 1055 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7238ca808804dddaf39389d335f4eae" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37143 } }
I20260807 00:44:32.412212 1057 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7238ca808804dddaf39389d335f4eae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7238ca808804dddaf39389d335f4eae" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37143 } } }
I20260807 00:44:32.412196 1055 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7238ca808804dddaf39389d335f4eae. Latest consensus state: current_term: 1 leader_uuid: "b7238ca808804dddaf39389d335f4eae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7238ca808804dddaf39389d335f4eae" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 37143 } } }
I20260807 00:44:32.412299 1057 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:32.412307 1055 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae [sys.catalog]: This master's current role is: LEADER
Aug 07 00:44:32 dist-test-slave-h6n0 krb5kdc[921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063472, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:32.412977 1065 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:32.413582 1065 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:32.415550 1065 catalog_manager.cc:1347] Generated new cluster ID: a616f645d18a49269b834407bbca85c5
I20260807 00:44:32.415620 1065 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:32.417382 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:32.404800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48507 (local address 127.1.116.190:37143)
0807 00:44:32.405007 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:32.405016 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:32.405047 (+ 31us) server_negotiation.cc:407] Connection header received
0807 00:44:32.405827 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:32.405836 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:32.405885 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:32.406013 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:32.407041 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:32.407968 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:32.409309 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:32.410097 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:32.410192 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:32.413750 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:32.413754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:32.413755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:32.413769 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:32.415898 (+ 2129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:32.416319 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:32.416321 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:32.416322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:32.416406 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:32.416603 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:32.416604 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:32.416604 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:32.416803 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:32.416908 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:32.417055 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 00:44:32.417167 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":96,"threads_started":1}
I20260807 00:44:32.418439 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:32.426237 1052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:32.419827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55795 (local address 127.1.116.190:37143)
0807 00:44:32.419866 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:32.419868 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:32.419946 (+ 78us) server_negotiation.cc:407] Connection header received
0807 00:44:32.420017 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:32.420018 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:32.420026 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:32.420073 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:32.420724 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:32.421179 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:32.422250 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:32.422434 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:32.422488 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:32.423850 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:32.423851 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:32.423852 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:32.423857 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:32.425200 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:32.425585 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:32.425585 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:32.425586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:32.425635 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:32.425832 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:32.425833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:32.425833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:32.425922 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:32.425993 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:32.426127 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 00:44:32.426186 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260807 00:44:32.447935 1065 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:32.448664 1065 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:32.457916 1065 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7238ca808804dddaf39389d335f4eae: Generated new TSK 0
I20260807 00:44:32.458117 1065 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:32.463943 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 942
2026-08-07T00:44:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1607 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:32 dist-test-slave-h6n0 krb5kdc[1094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:32 dist-test-slave-h6n0 krb5kdc[1094](info): set up 2 sockets
Aug 07 00:44:32 dist-test-slave-h6n0 krb5kdc[1094](info): commencing operation
krb5kdc: starting...
W20260807 00:44:33.719328 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:33 dist-test-slave-h6n0 krb5kdc[1094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:33.849071 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43701
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:43701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:33.939790 1109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:33.940059 1109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:33.940097 1109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:33.941313 1109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:33.941386 1109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:33.941412 1109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:33.941434 1109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:33.941460 1109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:33.943008 1109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:43701
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:33.943316 1109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:33.943562 1109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:33.948802 1120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:33.948839 1117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:33.948822 1116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:33.949479 1109 server_base.cc:1034] running on GCE node
I20260807 00:44:33.949769 1109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:33.950083 1109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:33.951233 1109 hybrid_clock.cc:648] HybridClock initialized: now 1786063473951208 us; error 46 us; skew 500 ppm
Aug 07 00:44:33 dist-test-slave-h6n0 krb5kdc[1094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063473, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:33.953990 1109 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:33.954605 1109 webserver.cc:460] Webserver started at http://127.1.116.190:45897/ using document root <none> and password file <none>
I20260807 00:44:33.954849 1109 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:33.954916 1109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:33.955078 1109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:33.956283 1109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "90dcf6e5f87a4288ac64049d781b1d38"
format_stamp: "Formatted at 2026-08-07 00:44:33 on dist-test-slave-h6n0"
server_key: "6132e1eb0082eb9c6b1e5b2c8422b481"
server_key_iv: "faaa24783005bb80eb545f6a8dc85b95"
server_key_version: "encryptionkey@0"
I20260807 00:44:33.956684 1109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "90dcf6e5f87a4288ac64049d781b1d38"
format_stamp: "Formatted at 2026-08-07 00:44:33 on dist-test-slave-h6n0"
server_key: "6132e1eb0082eb9c6b1e5b2c8422b481"
server_key_iv: "faaa24783005bb80eb545f6a8dc85b95"
server_key_version: "encryptionkey@0"
I20260807 00:44:33.958242 1109 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:33.959164 1127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:33.959388 1109 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260807 00:44:33.959496 1109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "90dcf6e5f87a4288ac64049d781b1d38"
format_stamp: "Formatted at 2026-08-07 00:44:33 on dist-test-slave-h6n0"
server_key: "6132e1eb0082eb9c6b1e5b2c8422b481"
server_key_iv: "faaa24783005bb80eb545f6a8dc85b95"
server_key_version: "encryptionkey@0"
I20260807 00:44:33.959579 1109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:33.970965 1109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:33.972927 1109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:33.973093 1109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:33.978473 1109 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:43701
I20260807 00:44:33.978590 1203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:43701 every 8 connection(s)
I20260807 00:44:33.978989 1109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:33.980108 1204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:33.982650 1204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38: Bootstrap starting.
I20260807 00:44:33.983266 1204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:33.983573 1204 log.cc:824] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:33.984400 1204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38: No bootstrap required, opened a new log
I20260807 00:44:33.984901 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1109
I20260807 00:44:33.985020 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:33.985268 1490 external_mini_cluster.cc:1428] Setting key 4b18cbc12aa8c1b641347106ae089eab
I20260807 00:44:33.986840 1204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dcf6e5f87a4288ac64049d781b1d38" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43701 } }
I20260807 00:44:33.987027 1204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:33.987067 1204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90dcf6e5f87a4288ac64049d781b1d38, State: Initialized, Role: FOLLOWER
I20260807 00:44:33.987237 1204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dcf6e5f87a4288ac64049d781b1d38" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43701 } }
I20260807 00:44:33.987351 1204 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:33.987386 1204 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:33.987440 1204 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:33.988219 1204 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dcf6e5f87a4288ac64049d781b1d38" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43701 } }
I20260807 00:44:33.988349 1204 leader_election.cc:302] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90dcf6e5f87a4288ac64049d781b1d38; no voters:
I20260807 00:44:33.988529 1204 leader_election.cc:288] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:33.988605 1209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:33.988783 1204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:33.988821 1209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [term 1 LEADER]: Becoming Leader. State: Replica: 90dcf6e5f87a4288ac64049d781b1d38, State: Running, Role: LEADER
I20260807 00:44:33.988962 1209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dcf6e5f87a4288ac64049d781b1d38" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43701 } }
I20260807 00:44:33.989367 1212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90dcf6e5f87a4288ac64049d781b1d38. Latest consensus state: current_term: 1 leader_uuid: "90dcf6e5f87a4288ac64049d781b1d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dcf6e5f87a4288ac64049d781b1d38" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43701 } } }
I20260807 00:44:33.989497 1212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:33.989714 1211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90dcf6e5f87a4288ac64049d781b1d38" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90dcf6e5f87a4288ac64049d781b1d38" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43701 } } }
I20260807 00:44:33.990151 1211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:33.990345 1219 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:33.990952 1219 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:33.993007 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:33.985825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48505 (local address 127.1.116.190:43701)
0807 00:44:33.986099 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:33.986109 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:33.986147 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:44:33.986797 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:33.986806 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:33.986880 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:33.987042 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:33.987352 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:33.989144 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:33.990382 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:33.991100 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:33.991564 (+ 464us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:33.991978 (+ 414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:33.991986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:33.991986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:33.992150 (+ 164us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:33.992189 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:33.992200 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:33.992288 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:33.992693 (+ 405us) server_negotiation.cc:299] Negotiation successful
0807 00:44:33.992770 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":126,"threads_started":1}
I20260807 00:44:33.993084 1219 catalog_manager.cc:1347] Generated new cluster ID: 44d0c0245e6541d2b044411b137b84f0
I20260807 00:44:33.993209 1219 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:33.994334 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:34.004776 1219 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:34.005393 1219 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:34.006750 1208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:33.998924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53639 (local address 127.1.116.190:43701)
0807 00:44:33.998976 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:33.998978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:34.003221 (+ 4243us) server_negotiation.cc:407] Connection header received
0807 00:44:34.003295 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:34.003296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:34.003313 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:34.003374 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:34.003551 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:34.004213 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:34.005252 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:34.005585 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:34.005676 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:34.006411 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:34.006413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:34.006413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:34.006418 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:34.006436 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:34.006441 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:34.006505 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:34.006594 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 00:44:34.006663 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:44:34.009742 1219 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90dcf6e5f87a4288ac64049d781b1d38: Generated new TSK 0
I20260807 00:44:34.009909 1219 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:34.012537 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1109
2026-08-07T00:44:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1550 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:34 dist-test-slave-h6n0 krb5kdc[1256](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:34 dist-test-slave-h6n0 krb5kdc[1256](info): set up 2 sockets
Aug 07 00:44:34 dist-test-slave-h6n0 krb5kdc[1256](info): commencing operation
krb5kdc: starting...
W20260807 00:44:35.264657 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:35 dist-test-slave-h6n0 krb5kdc[1256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:35.390895 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43349
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:38061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:43349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:35.488312 1277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:35.488566 1277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:35.488595 1277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:35.489888 1277 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:35.489933 1277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:35.489954 1277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:35.489967 1277 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:35.490020 1277 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:35.491637 1277 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:38061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:43349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:43349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:35.491917 1277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:35.492192 1277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:35.497766 1290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:35.497964 1283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:35.497766 1287 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:35.498119 1277 server_base.cc:1034] running on GCE node
I20260807 00:44:35.498436 1277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:35.498858 1277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:35.500059 1277 hybrid_clock.cc:648] HybridClock initialized: now 1786063475500026 us; error 57 us; skew 500 ppm
Aug 07 00:44:35 dist-test-slave-h6n0 krb5kdc[1256](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063475, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:35.503115 1277 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:35.503645 1277 webserver.cc:460] Webserver started at http://127.1.116.190:36585/ using document root <none> and password file <none>
I20260807 00:44:35.503891 1277 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:35.503957 1277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:35.504115 1277 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:35.505285 1277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "8d7e0ef9ab6a42498817235a2268d14d"
format_stamp: "Formatted at 2026-08-07 00:44:35 on dist-test-slave-h6n0"
server_key: "aaa38258c181d2d7d7c60220ecef0d42"
server_key_iv: "99ab66cccfc64995141070ac780e43f8"
server_key_version: "encryptionkey@0"
I20260807 00:44:35.505663 1277 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "8d7e0ef9ab6a42498817235a2268d14d"
format_stamp: "Formatted at 2026-08-07 00:44:35 on dist-test-slave-h6n0"
server_key: "aaa38258c181d2d7d7c60220ecef0d42"
server_key_iv: "99ab66cccfc64995141070ac780e43f8"
server_key_version: "encryptionkey@0"
I20260807 00:44:35.507196 1277 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260807 00:44:35.507999 1300 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:35.508203 1277 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:35.508323 1277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "8d7e0ef9ab6a42498817235a2268d14d"
format_stamp: "Formatted at 2026-08-07 00:44:35 on dist-test-slave-h6n0"
server_key: "aaa38258c181d2d7d7c60220ecef0d42"
server_key_iv: "99ab66cccfc64995141070ac780e43f8"
server_key_version: "encryptionkey@0"
I20260807 00:44:35.508396 1277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:35.511854 1277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:35.514091 1277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:35.514242 1277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:35.518931 1277 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:43349
I20260807 00:44:35.518982 1375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:43349 every 8 connection(s)
I20260807 00:44:35.519340 1277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:35.520511 1376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:35.523289 1376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d: Bootstrap starting.
I20260807 00:44:35.523931 1376 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:35.524217 1376 log.cc:824] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:35.524927 1376 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d: No bootstrap required, opened a new log
I20260807 00:44:35.526783 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1277
I20260807 00:44:35.526757 1376 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7e0ef9ab6a42498817235a2268d14d" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43349 } }
I20260807 00:44:35.526916 1376 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:44:35.526937 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:35.526965 1376 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d7e0ef9ab6a42498817235a2268d14d, State: Initialized, Role: FOLLOWER
I20260807 00:44:35.527165 1490 external_mini_cluster.cc:1428] Setting key 8089a872ebabf8fdfdec280ac6c52768
I20260807 00:44:35.527156 1376 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7e0ef9ab6a42498817235a2268d14d" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43349 } }
I20260807 00:44:35.527264 1376 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:44:35.527312 1376 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:44:35.527364 1376 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:35.528182 1376 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7e0ef9ab6a42498817235a2268d14d" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43349 } }
I20260807 00:44:35.528301 1376 leader_election.cc:302] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d7e0ef9ab6a42498817235a2268d14d; no voters:
I20260807 00:44:35.528481 1376 leader_election.cc:288] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:44:35.528568 1386 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:35.528791 1386 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [term 1 LEADER]: Becoming Leader. State: Replica: 8d7e0ef9ab6a42498817235a2268d14d, State: Running, Role: LEADER
I20260807 00:44:35.528789 1376 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:35.530215 1386 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7e0ef9ab6a42498817235a2268d14d" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43349 } }
I20260807 00:44:35.530942 1388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d7e0ef9ab6a42498817235a2268d14d. Latest consensus state: current_term: 1 leader_uuid: "8d7e0ef9ab6a42498817235a2268d14d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7e0ef9ab6a42498817235a2268d14d" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43349 } } }
I20260807 00:44:35.531366 1388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:35.531056 1387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d7e0ef9ab6a42498817235a2268d14d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d7e0ef9ab6a42498817235a2268d14d" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 43349 } } }
I20260807 00:44:35.531661 1387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:35.531838 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:35.527688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49361 (local address 127.1.116.190:43349)
0807 00:44:35.528267 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:35.528279 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:35.528318 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:44:35.528454 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:35.528462 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:35.528497 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:35.528987 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:35.531147 (+ 2160us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:35.531151 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:35.531151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:35.531157 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:35.531211 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:35.531229 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:35.531319 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:35.531551 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 00:44:35.531604 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":428,"threads_started":1}
W20260807 00:44:35.531858 1407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:35.531924 1407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:35.531975 1411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:35.532173 1411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:35.533114 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:35.533871 1411 catalog_manager.cc:1347] Generated new cluster ID: f7d620948b41457da86613f49d5bd9d4
I20260807 00:44:35.533942 1411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:35.536428 1385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:35.535336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39923 (local address 127.1.116.190:43349)
0807 00:44:35.535390 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:35.535392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:35.535404 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:44:35.535487 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:35.535488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:35.535494 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:35.535553 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:35.536196 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:35.536197 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:35.536198 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:44:35.536202 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:35.536218 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:44:35.536223 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:35.536281 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:35.536355 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 00:44:35.536367 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:44:35.541982 1411 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:35.542169 1411 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:35.545899 1411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d7e0ef9ab6a42498817235a2268d14d: Generated new TSK 0
I20260807 00:44:35.546073 1411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:44:35.553385 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1277
2026-08-07T00:44:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1541 ms)
[----------] 18 tests from AuthTokenIssuingTest (29657 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:35 dist-test-slave-h6n0 krb5kdc[1425](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:35 dist-test-slave-h6n0 krb5kdc[1425](info): set up 2 sockets
Aug 07 00:44:35 dist-test-slave-h6n0 krb5kdc[1425](info): commencing operation
krb5kdc: starting...
W20260807 00:44:36.808825 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:36 dist-test-slave-h6n0 krb5kdc[1425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:36.940994 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:42541
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:42541,127.1.116.189:37065,127.1.116.188:43041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:37.036201 1445 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:44:37.036489 1445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:37.036509 1445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:37.036535 1445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:37.037942 1445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:37.038002 1445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:37.038017 1445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:37.038031 1445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:37.038096 1445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:37.039901 1445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:42541,127.1.116.189:37065,127.1.116.188:43041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:42541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:37.040261 1445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:37.040560 1445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:37.046193 1455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:37.046240 1452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:37.046192 1451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:37.046772 1445 server_base.cc:1034] running on GCE node
I20260807 00:44:37.047080 1445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:37.047439 1445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:37.048640 1445 hybrid_clock.cc:648] HybridClock initialized: now 1786063477048613 us; error 44 us; skew 500 ppm
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:37.054813 1445 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:37.055272 1445 webserver.cc:460] Webserver started at http://127.1.116.190:46007/ using document root <none> and password file <none>
I20260807 00:44:37.055512 1445 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:37.055568 1445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:37.055706 1445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:37.057018 1445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "aa42c30817094621a34709a9d8490697"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "c49088a22856f5c8a3a362f598606e26"
server_key_iv: "2262c14c00beb5160d6e9bae0f61f170"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.057402 1445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "aa42c30817094621a34709a9d8490697"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "c49088a22856f5c8a3a362f598606e26"
server_key_iv: "2262c14c00beb5160d6e9bae0f61f170"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.059036 1445 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:37.060015 1468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:37.060340 1445 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:37.060484 1445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "aa42c30817094621a34709a9d8490697"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "c49088a22856f5c8a3a362f598606e26"
server_key_iv: "2262c14c00beb5160d6e9bae0f61f170"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.060568 1445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:37.085278 1445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:37.087584 1445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:37.087740 1445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:37.092125 1445 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:42541
I20260807 00:44:37.092226 1550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:42541 every 8 connection(s)
I20260807 00:44:37.092549 1445 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:37.093743 1551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:37.095618 1551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:37.097395 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1445
I20260807 00:44:37.097532 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:37.097748 1490 external_mini_cluster.cc:1428] Setting key eebaa288027cdfe2898948dfb24a440c
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.189@KRBTEST.COM; defaulting to no policy
I20260807 00:44:37.106079 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.096311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42541 (local address 127.0.0.1:46086)
0807 00:44:37.096591 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.096631 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.097289 (+ 658us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.097685 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.097688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.098034 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.098202 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.098207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.099368 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.099369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.100070 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.100072 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.100640 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.100641 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.100711 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.101303 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.101312 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.102864 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.104684 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.104690 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.104693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.104887 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.105131 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.105132 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.105132 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.105227 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.105580 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.105586 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.105648 (+ 62us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.105756 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.105896 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":191,"threads_started":1}
I20260807 00:44:37.106099 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.096401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46086 (local address 127.1.116.190:42541)
0807 00:44:37.096607 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.096616 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.096731 (+ 115us) server_negotiation.cc:407] Connection header received
0807 00:44:37.097461 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.097463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.097483 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.097677 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.098323 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.099262 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.100163 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.100496 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.100654 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.102964 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.102972 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.102972 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.102979 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.104583 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.104980 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.104983 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.104983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.105062 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.105290 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.105291 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.105291 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.105504 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.105590 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.105875 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.106016 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":117,"threads_started":1}
I20260807 00:44:37.107002 1551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:37.107373 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.107251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47182)
0807 00:44:37.107300 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.107346 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260807 00:44:37.107496 1469 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
I20260807 00:44:37.108003 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.107921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47188)
0807 00:44:37.107958 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.107983 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260807 00:44:37.108119 1551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:37065: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab.
Entry for principal kudu/127.1.116.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab.
I20260807 00:44:37.125554 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:37065
--webserver_interface=127.1.116.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab
--principal=kudu/127.1.116.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:42541,127.1.116.189:37065,127.1.116.188:43041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:44:37.157359 1551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } attempt: 1
I20260807 00:44:37.157895 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.157762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47204)
0807 00:44:37.157828 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.157866 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 00:44:37.158218 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.158127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47212)
0807 00:44:37.158170 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.158195 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 00:44:37.158313 1551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:37065: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
W20260807 00:44:37.219064 1566 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:44:37.219322 1566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:37.219340 1566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:37.219362 1566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:37.220530 1566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:37.220575 1566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:37.220590 1566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:37.220602 1566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:37.220614 1566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:37.222234 1566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:42541,127.1.116.189:37065,127.1.116.188:43041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab
--principal=kudu/127.1.116.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:37065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.116.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:37.222612 1566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:37.222893 1566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:44:37.226468 1551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } attempt: 2
I20260807 00:44:37.227268 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.227020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47214)
0807 00:44:37.227132 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.227182 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260807 00:44:37.227891 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.227780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47222)
0807 00:44:37.227828 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.227859 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260807 00:44:37.228058 1551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:37065: Network error: Client connection negotiation failed: client connection to 127.1.116.189:37065: connect: Connection refused (error 111)
W20260807 00:44:37.228395 1576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:37.228381 1578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:37.228513 1575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:37.229177 1566 server_base.cc:1034] running on GCE node
I20260807 00:44:37.229475 1566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:37.229815 1566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:37.230988 1566 hybrid_clock.cc:648] HybridClock initialized: now 1786063477230968 us; error 39 us; skew 500 ppm
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:37.233886 1566 init.cc:377] Logged in from keytab as kudu/127.1.116.189@KRBTEST.COM (short username kudu)
I20260807 00:44:37.234424 1566 webserver.cc:460] Webserver started at http://127.1.116.189:46025/ using document root <none> and password file <none>
I20260807 00:44:37.234709 1566 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:37.234768 1566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:37.234876 1566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:37.236101 1566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data/instance:
uuid: "b71b998e754c487ea8a1c50918af202f"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "73c046ebec81c7b449d1b5dfd4631e21"
server_key_iv: "4c412c02792ec2963e2f9bb81768922d"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.236521 1566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/wal/instance:
uuid: "b71b998e754c487ea8a1c50918af202f"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "73c046ebec81c7b449d1b5dfd4631e21"
server_key_iv: "4c412c02792ec2963e2f9bb81768922d"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.238189 1566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260807 00:44:37.239244 1588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:37.239598 1566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:37.239720 1566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/wal
uuid: "b71b998e754c487ea8a1c50918af202f"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "73c046ebec81c7b449d1b5dfd4631e21"
server_key_iv: "4c412c02792ec2963e2f9bb81768922d"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.239807 1566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:37.250269 1566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:37.252753 1566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:37.252914 1566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:37.258347 1566 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.189:37065
I20260807 00:44:37.258437 1673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.189:37065 every 8 connection(s)
I20260807 00:44:37.258826 1566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
I20260807 00:44:37.259935 1674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:37.261631 1674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:37.262142 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1566
I20260807 00:44:37.262243 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-1/wal/instance
I20260807 00:44:37.262471 1490 external_mini_cluster.cc:1428] Setting key 59ea6cc1c6abed9e63fb9ff5fe49340b
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.188@KRBTEST.COM; defaulting to no policy
I20260807 00:44:37.274129 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.262471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42541 (local address 127.0.0.1:46090)
0807 00:44:37.262723 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.262780 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.263556 (+ 776us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.263903 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.263908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.264326 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.264650 (+ 324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.264656 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.265839 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.265840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.266770 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.266773 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.267374 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.267376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.267492 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.268140 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.268149 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.270014 (+ 1865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.272415 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.272423 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.272425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.272719 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.272982 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.272982 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.272983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.273163 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.273520 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.273527 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.273649 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.273754 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.273881 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":146,"threads_started":1}
I20260807 00:44:37.274195 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.262493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.1.116.190:42541)
0807 00:44:37.262588 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.262590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.262912 (+ 322us) server_negotiation.cc:407] Connection header received
0807 00:44:37.263746 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.263747 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.263764 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.263829 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.264800 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.265621 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.266870 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.267245 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.267333 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.270130 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.270132 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.270133 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.270143 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.272292 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.272822 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.272823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.272824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.272892 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.273246 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.273247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.273247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.273421 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.273502 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.273935 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.274108 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:44:37.274806 1674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.188@KRBTEST.COM" created.
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:44:37.283430 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.275098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47224)
0807 00:44:37.275163 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.275176 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.275225 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.275567 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.275568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.275778 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.275863 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.275866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.277027 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.277027 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.277631 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.277633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.278241 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.278243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.278335 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.278840 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.278846 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.280311 (+ 1465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.282332 (+ 2021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.282333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.282333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.282515 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.282760 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.282761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.282762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.282817 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.283100 (+ 283us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.283101 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.283141 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.283205 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.283319 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260807 00:44:37.283524 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.275178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47224 (local address 127.1.116.189:37065)
0807 00:44:37.275387 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.275392 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.275409 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:44:37.275429 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.275431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.275443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.275564 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.275950 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.276921 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.277720 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.278148 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.278212 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.280409 (+ 2197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.280417 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.280417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.280428 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.282223 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.282635 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.282638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.282638 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.282694 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.282884 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.282885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.282885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.283022 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.283108 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.283239 (+ 131us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.283385 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":79,"server-negotiator.queue_time_us":180,"thread_start_us":136,"threads_started":1}
I20260807 00:44:37.284072 1674 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:37.284495 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.284310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38228)
0807 00:44:37.284380 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.284450 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 00:44:37.284777 1596 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
I20260807 00:44:37.285286 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.285157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38230)
0807 00:44:37.285228 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.285261 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 00:44:37.285429 1674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:43041: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab.
Entry for principal kudu/127.1.116.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab.
I20260807 00:44:37.293469 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:43041
--webserver_interface=127.1.116.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab
--principal=kudu/127.1.116.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:42541,127.1.116.189:37065,127.1.116.188:43041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:44:37.319253 1551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } attempt: 3
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:44:37.327498 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.319632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47230)
0807 00:44:37.319694 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.319708 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.319764 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.320089 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.320091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.320255 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.320336 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.320338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.321361 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.321362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.321932 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.321933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.322461 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.322462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.322589 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.323198 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.323208 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.324480 (+ 1272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.326467 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.326468 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.326468 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.326691 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.326908 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.326908 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.326909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.326965 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.327261 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.327262 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.327301 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.327358 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.327431 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260807 00:44:37.327596 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.319748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47230 (local address 127.1.116.189:37065)
0807 00:44:37.319838 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.319841 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.319857 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:44:37.319899 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.319901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.319919 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.320033 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.320436 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.321262 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.322021 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.322378 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.322460 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.324579 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.324581 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.324581 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.324590 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.326375 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.326765 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.326766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.326767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.326833 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.327047 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.327047 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.327048 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.327190 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.327271 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.327403 (+ 132us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.327487 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:44:37.328117 1551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:37.328540 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.328404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38246)
0807 00:44:37.328468 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.328513 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 00:44:37.329043 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.328945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38252)
0807 00:44:37.328994 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.329023 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260807 00:44:37.329169 1551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:43041: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
I20260807 00:44:37.334671 1674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } attempt: 1
I20260807 00:44:37.335108 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.334964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38256)
0807 00:44:37.335035 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.335074 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 00:44:37.335685 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.335569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38260)
0807 00:44:37.335637 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.335666 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 00:44:37.335914 1674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:43041: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
I20260807 00:44:37.360342 1551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } attempt: 1
I20260807 00:44:37.360836 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.360610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38272)
0807 00:44:37.360724 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.360786 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 00:44:37.361258 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.361147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38276)
0807 00:44:37.361199 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.361234 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 00:44:37.361385 1551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:43041: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
W20260807 00:44:37.388006 1685 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:44:37.388257 1685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:37.388275 1685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:37.388295 1685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:37.389446 1685 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:37.389494 1685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:37.389508 1685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:37.389521 1685 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:37.389534 1685 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:37.391116 1685 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:42541,127.1.116.189:37065,127.1.116.188:43041
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab
--principal=kudu/127.1.116.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:43041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.116.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:37.391412 1685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:37.391685 1685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:37.396942 1694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:37.396957 1691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:37.397054 1692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:37.397332 1685 server_base.cc:1034] running on GCE node
I20260807 00:44:37.397574 1685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:37.397890 1685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:37.399041 1685 hybrid_clock.cc:648] HybridClock initialized: now 1786063477399024 us; error 36 us; skew 500 ppm
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:37.401754 1685 init.cc:377] Logged in from keytab as kudu/127.1.116.188@KRBTEST.COM (short username kudu)
I20260807 00:44:37.402302 1685 webserver.cc:460] Webserver started at http://127.1.116.188:44397/ using document root <none> and password file <none>
I20260807 00:44:37.402623 1685 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:37.402695 1685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:37.402854 1685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:37.404063 1674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } attempt: 2
I20260807 00:44:37.404470 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.404349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38284)
0807 00:44:37.404411 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.404445 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260807 00:44:37.404469 1685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data/instance:
uuid: "eaf2626526e1442dba4d1d177789a0dc"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "e2d36b65dceb198c2f6940e8acef7872"
server_key_iv: "fbe3b7763d51c83456debd9ad4ee59f7"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.404914 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.404823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38300)
0807 00:44:37.404865 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.404896 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260807 00:44:37.405035 1685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/wal/instance:
uuid: "eaf2626526e1442dba4d1d177789a0dc"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "e2d36b65dceb198c2f6940e8acef7872"
server_key_iv: "fbe3b7763d51c83456debd9ad4ee59f7"
server_key_version: "encryptionkey@0"
W20260807 00:44:37.405143 1674 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:43041: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
I20260807 00:44:37.406697 1685 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 00:44:37.407671 1702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:37.407996 1685 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:37.408142 1685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/wal
uuid: "eaf2626526e1442dba4d1d177789a0dc"
format_stamp: "Formatted at 2026-08-07 00:44:37 on dist-test-slave-h6n0"
server_key: "e2d36b65dceb198c2f6940e8acef7872"
server_key_iv: "fbe3b7763d51c83456debd9ad4ee59f7"
server_key_version: "encryptionkey@0"
I20260807 00:44:37.408234 1685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:37.436563 1551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } attempt: 2
I20260807 00:44:37.437124 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.436892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38314)
0807 00:44:37.436990 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.437064 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 00:44:37.437566 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.437470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38326)
0807 00:44:37.437520 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.437549 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 00:44:37.437685 1551 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:43041: Network error: Client connection negotiation failed: client connection to 127.1.116.188:43041: connect: Connection refused (error 111)
I20260807 00:44:37.459069 1685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:37.461383 1685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:37.461552 1685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:37.466889 1685 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.188:43041
I20260807 00:44:37.466969 1787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.188:43041 every 8 connection(s)
I20260807 00:44:37.467361 1685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
I20260807 00:44:37.468494 1789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:37.470216 1789 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:37.470376 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 1685
I20260807 00:44:37.470517 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786063190385254-1490-0/minicluster-data/master-2/wal/instance
I20260807 00:44:37.470752 1490 external_mini_cluster.cc:1428] Setting key c8f9414ff6c133a605436ac286c55258
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:44:37.480278 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.471026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46092 (local address 127.1.116.190:42541)
0807 00:44:37.471091 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.471093 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.471412 (+ 319us) server_negotiation.cc:407] Connection header received
0807 00:44:37.472083 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.472085 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.472099 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.472170 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.472782 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.473292 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.474232 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.474429 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.474515 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.477057 (+ 2542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.477058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.477059 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.477066 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.478778 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.479280 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.479281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.479282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.479349 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.479594 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.479595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.479595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.479735 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.479807 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.480083 (+ 276us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.480165 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260807 00:44:37.480538 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.471209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.1.116.188:43041)
0807 00:44:37.471420 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.471427 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.471468 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:44:37.471978 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.471990 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.472016 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.472120 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.472790 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.473633 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.474369 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.474812 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.474904 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.477521 (+ 2617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.477529 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.477530 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.477538 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.479226 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.479568 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.479571 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.479571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.479635 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.479833 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.479834 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.479834 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.479983 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.480154 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.480200 (+ 46us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.480326 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":170,"thread_start_us":95,"threads_started":1}
I20260807 00:44:37.480556 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.471012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:42541 (local address 127.0.0.1:46092)
0807 00:44:37.471255 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.471315 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.471958 (+ 643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.472212 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.472216 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.472504 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.472686 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.472690 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.473373 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.473375 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.474118 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.474121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.474572 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.474574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.474729 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.475286 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.475299 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.476957 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.478873 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.478881 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.478883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.479163 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.479415 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.479416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.479416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.479525 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.479809 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.479817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.479949 (+ 132us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.480059 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.480254 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260807 00:44:37.481348 1789 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:37.489334 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.481646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:37065 (local address 127.0.0.1:47234)
0807 00:44:37.481882 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.481895 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.481956 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.482116 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.482117 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.482294 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.482327 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.482329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.483017 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.483018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.483572 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.483574 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.484040 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.484041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.484101 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.484523 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.484527 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.485962 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.488185 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.488186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.488187 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.488393 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.488647 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.488648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.488648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.488713 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.488992 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.488993 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.489039 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.489133 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.489235 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215}
I20260807 00:44:37.489362 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.481788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47234 (local address 127.1.116.189:37065)
0807 00:44:37.481836 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.481838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.481955 (+ 117us) server_negotiation.cc:407] Connection header received
0807 00:44:37.482038 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.482039 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.482047 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.482091 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.482402 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.482935 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.483652 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.483968 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.484023 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.486080 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.486082 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.486082 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.486091 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.488039 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.488490 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.488492 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.488492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.488562 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.488781 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.488782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.488783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.488918 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.489000 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.489161 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.489246 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 00:44:37.489671 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.481649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46094 (local address 127.1.116.190:42541)
0807 00:44:37.481698 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.481700 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.481708 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:44:37.481732 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.481732 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.481748 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.481818 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.482583 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.483104 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.483892 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.484123 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.484203 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.486801 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.486803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.486803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.486810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.488328 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.488698 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.488699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.488700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.488757 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.489064 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.489066 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.489066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.489241 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.489433 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.489476 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.489563 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260807 00:44:37.490576 1789 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:44:37.496304 1674 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } attempt: 3
I20260807 00:44:37.497984 1795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.490895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38350 (local address 127.1.116.188:43041)
0807 00:44:37.490930 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.490932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.491076 (+ 144us) server_negotiation.cc:407] Connection header received
0807 00:44:37.491093 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.491094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.491107 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.491148 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.491544 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.492014 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.492690 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.492918 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.492996 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.495112 (+ 2116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.495113 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.495114 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.495121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.496826 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.497180 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.497181 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.497182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.497234 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.497410 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.497410 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.497410 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.497537 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.497625 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.497786 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.497876 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260807 00:44:37.498051 1793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.490865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38350)
0807 00:44:37.490912 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.490923 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.490971 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.491232 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.491233 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.491396 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.491435 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.491437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.492100 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.492101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.492590 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.492591 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.492997 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.492997 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.493051 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.493539 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.493544 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.494935 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.496909 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.496910 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.496911 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.497074 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.497298 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.497298 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.497299 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.497350 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.497630 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.497631 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.497678 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.497759 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.497929 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29,"mutex_wait_us":74}
I20260807 00:44:37.499024 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.490770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.1.116.189:37065)
0807 00:44:37.490814 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.490816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.490875 (+ 59us) server_negotiation.cc:407] Connection header received
0807 00:44:37.490970 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.490971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.490983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.491037 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.491773 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.492244 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.492915 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.493138 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.493213 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.495737 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.495739 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.495739 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.495746 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.497723 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.498224 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.498224 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.498225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.498283 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.498489 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.498490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.498490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.498634 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.498794 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.498837 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.498913 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:44:37.500974 1789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: Bootstrap starting.
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:44:37.502033 1789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:37.502555 1789 log.cc:824] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:37.503590 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:37.503635 1789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: No bootstrap required, opened a new log
I20260807 00:44:37.505363 1676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.496576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38364)
0807 00:44:37.496626 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.496636 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.496677 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.497059 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.497060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.497349 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.497411 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.497414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.498575 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.498576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.499449 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.499452 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.499965 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.499966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.500028 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.500475 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.500481 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.501869 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.504208 (+ 2339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.504209 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.504210 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.504433 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.504665 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.504666 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.504666 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.504726 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.505025 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.505026 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.505072 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.505164 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.505272 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260807 00:44:37.505409 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.496640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.1.116.188:43041)
0807 00:44:37.496894 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.496896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.496905 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:44:37.496925 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.496926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.496937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.497081 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.497512 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.498409 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.499553 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.499881 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.499950 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.501980 (+ 2030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.501981 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.501982 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.501990 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.504093 (+ 2103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.504542 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.504543 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.504543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.504610 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.504800 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.504801 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.504802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.504937 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.505030 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.505194 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.505274 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":223,"thread_start_us":155,"threads_started":1}
I20260807 00:44:37.505815 1789 raft_consensus.cc:348] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.504879 1520 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "aa42c30817094621a34709a9d8490697"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46094
I20260807 00:44:37.505999 1789 raft_consensus.cc:729] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eaf2626526e1442dba4d1d177789a0dc, State: Initialized, Role: FOLLOWER
I20260807 00:44:37.506099 1789 consensus_queue.cc:260] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.506495 1789 sys_catalog.cc:564] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:37.506464 1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.506656 1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:37.508247 1674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: Bootstrap starting.
W20260807 00:44:37.508749 1818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:37.508868 1818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:37.509166 1674 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:37.509557 1674 log.cc:824] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:37.510815 1674 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: No bootstrap required, opened a new log
I20260807 00:44:37.509117 1685 master_runner.cc:186] Error getting master registration for 127.1.116.190:42541: OK, instance_id { permanent_uuid: "aa42c30817094621a34709a9d8490697" instance_seqno: 1786063477090912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:44:37.511500 1685 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:44:37.513499 1674 raft_consensus.cc:348] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.513739 1674 raft_consensus.cc:729] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b71b998e754c487ea8a1c50918af202f, State: Initialized, Role: FOLLOWER
I20260807 00:44:37.513859 1674 consensus_queue.cc:260] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.514271 1674 sys_catalog.cc:564] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:37.514300 1825 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.514418 1825 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [sys.catalog]: This master's current role is: FOLLOWER
W20260807 00:44:37.516372 1843 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:37.516427 1843 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:37.516669 1566 master_runner.cc:186] Error getting master registration for 127.1.116.190:42541: OK, instance_id { permanent_uuid: "aa42c30817094621a34709a9d8490697" instance_seqno: 1786063477090912 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:44:37.519403 1566 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:44:37.536782 1551 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } attempt: 3
Aug 07 00:44:37 dist-test-slave-h6n0 krb5kdc[1425](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063477, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:44:37.545207 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.537414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.1.116.188:43041)
0807 00:44:37.537473 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.537475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.537482 (+ 7us) server_negotiation.cc:407] Connection header received
0807 00:44:37.537507 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.537508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.537519 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.537584 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.537983 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.538512 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.539576 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.539807 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.539886 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.542192 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.542194 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.542194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.542201 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.543987 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.544370 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.544372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.544372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.544428 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.544625 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.544626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.544626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.544800 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.544865 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.545023 (+ 158us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.545103 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 00:44:37.545207 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.537193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:43041 (local address 127.0.0.1:38374)
0807 00:44:37.537273 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:37.537285 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:37.537332 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:37.537605 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:37.537606 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:37.537834 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:37.537900 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.537903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.538650 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.538651 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.539494 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:37.539496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.539885 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.539886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:37.539946 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.540457 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:37.540462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:37.542100 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:37.544069 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.544070 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.544070 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.544310 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.544502 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:37.544503 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:37.544503 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:37.544565 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:37.544860 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:37.544861 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:37.544908 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:44:37.544995 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:44:37.545103 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 00:44:37.547645 1551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697: Bootstrap starting.
I20260807 00:44:37.548549 1551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:37.548903 1551 log.cc:824] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:37.549815 1551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697: No bootstrap required, opened a new log
I20260807 00:44:37.551347 1551 raft_consensus.cc:348] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.551508 1551 raft_consensus.cc:729] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa42c30817094621a34709a9d8490697, State: Initialized, Role: FOLLOWER
I20260807 00:44:37.551601 1551 consensus_queue.cc:260] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.551978 1551 sys_catalog.cc:564] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:37.552073 1520 raft_consensus.cc:482] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 00:44:37.552126 1520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:37.553161 1520 raft_consensus.cc:504] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.553488 1520 leader_election.cc:288] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [CANDIDATE]: Term 1 election: Requested vote from peers b71b998e754c487ea8a1c50918af202f (127.1.116.189:37065), eaf2626526e1442dba4d1d177789a0dc (127.1.116.188:43041)
I20260807 00:44:37.554180 1634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa42c30817094621a34709a9d8490697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b71b998e754c487ea8a1c50918af202f"
I20260807 00:44:37.554371 1634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:37.555370 1634 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa42c30817094621a34709a9d8490697 in term 1.
I20260807 00:44:37.555770 1748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "aa42c30817094621a34709a9d8490697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eaf2626526e1442dba4d1d177789a0dc"
I20260807 00:44:37.555888 1469 leader_election.cc:302] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aa42c30817094621a34709a9d8490697, b71b998e754c487ea8a1c50918af202f; no voters:
I20260807 00:44:37.555908 1748 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:37.556986 1748 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aa42c30817094621a34709a9d8490697 in term 1.
I20260807 00:44:37.551929 1848 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.557410 1848 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:37.557524 1848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:37.557624 1848 raft_consensus.cc:686] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [term 1 LEADER]: Becoming Leader. State: Replica: aa42c30817094621a34709a9d8490697, State: Running, Role: LEADER
I20260807 00:44:37.557757 1848 consensus_queue.cc:237] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } }
I20260807 00:44:37.558343 1848 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa42c30817094621a34709a9d8490697. Latest consensus state: current_term: 1 leader_uuid: "aa42c30817094621a34709a9d8490697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.558492 1848 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:37.558780 1445 master_runner.cc:263] No leader master found from master aa42c30817094621a34709a9d8490697
I20260807 00:44:37.558830 1445 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:44:37.559248 1873 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:37.560302 1873 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:37.561856 1748 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [term 1 FOLLOWER]: Refusing update from remote peer aa42c30817094621a34709a9d8490697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:37.562177 1870 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:37.563869 1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa42c30817094621a34709a9d8490697. Latest consensus state: current_term: 1 leader_uuid: "aa42c30817094621a34709a9d8490697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.564064 1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:37.564067 1634 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [term 1 FOLLOWER]: Refusing update from remote peer aa42c30817094621a34709a9d8490697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:37.564343 1848 consensus_queue.cc:1035] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:37.565672 1825 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [sys.catalog]: SysCatalogTable state changed. Reason: New leader aa42c30817094621a34709a9d8490697. Latest consensus state: current_term: 1 leader_uuid: "aa42c30817094621a34709a9d8490697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.565796 1825 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:37.567838 1848 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa42c30817094621a34709a9d8490697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.567932 1848 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:37.568428 1799 sys_catalog.cc:455] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa42c30817094621a34709a9d8490697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.568529 1799 sys_catalog.cc:458] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:37.568629 1876 mvcc.cc:205] Tried to move back new op lower bound from 7315716004090884096 to 7315716004077756416. Current Snapshot: MvccSnapshot[applied={T|T < 7315716004090884096}]
I20260807 00:44:37.568746 1882 mvcc.cc:205] Tried to move back new op lower bound from 7315716004090884096 to 7315716004077756416. Current Snapshot: MvccSnapshot[applied={T|T < 7315716004090884096}]
I20260807 00:44:37.569844 1870 sys_catalog.cc:455] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "aa42c30817094621a34709a9d8490697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.569936 1870 sys_catalog.cc:458] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:37.570361 1825 sys_catalog.cc:455] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "aa42c30817094621a34709a9d8490697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aa42c30817094621a34709a9d8490697" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 42541 } } peers { permanent_uuid: "b71b998e754c487ea8a1c50918af202f" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 37065 } } peers { permanent_uuid: "eaf2626526e1442dba4d1d177789a0dc" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 43041 } } }
I20260807 00:44:37.570479 1825 sys_catalog.cc:458] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:37.570590 1888 mvcc.cc:205] Tried to move back new op lower bound from 7315716004090884096 to 7315716004077756416. Current Snapshot: MvccSnapshot[applied={T|T < 7315716004090884096}]
I20260807 00:44:37.579217 1873 catalog_manager.cc:1347] Generated new cluster ID: b28d5b6519744ee184b9845a5a2805e6
I20260807 00:44:37.579288 1873 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:37.587608 1796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.564591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38376 (local address 127.1.116.188:43041)
0807 00:44:37.564634 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.564636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.570897 (+ 6261us) server_negotiation.cc:407] Connection header received
0807 00:44:37.571009 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.571011 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.571023 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.571055 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.571903 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.572719 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.573843 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.574140 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.574184 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.576239 (+ 2055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.576241 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.576241 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.576249 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.578102 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.579628 (+ 1526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.579630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.579631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.579723 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.586610 (+ 6887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.586611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.586612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.586769 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.586888 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.587407 (+ 519us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.587512 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260807 00:44:37.590801 1679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.562860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47250 (local address 127.1.116.189:37065)
0807 00:44:37.562910 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.562913 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.570686 (+ 7773us) server_negotiation.cc:407] Connection header received
0807 00:44:37.570803 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.570805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.570816 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.570847 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.571697 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.572457 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.574596 (+ 2139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.574899 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.574941 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.587082 (+ 12141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.587083 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.587084 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.587090 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.589194 (+ 2104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.589639 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.589640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.589641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.589700 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.589893 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.589894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.589894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.590030 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.590118 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.590612 (+ 494us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.590699 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260807 00:44:37.595216 1557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:37.564339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46110 (local address 127.1.116.190:42541)
0807 00:44:37.564394 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.564396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.573644 (+ 9248us) server_negotiation.cc:407] Connection header received
0807 00:44:37.573745 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.573747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.573758 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.573788 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.577279 (+ 3491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.578014 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.587907 (+ 9893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:37.588181 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:37.588228 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:37.591965 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:37.591967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:37.591967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:37.591975 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:37.593680 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.594089 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.594091 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.594091 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.594148 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:37.594357 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:37.594358 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:37.594358 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:37.594491 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:37.594628 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:37.595037 (+ 409us) server_negotiation.cc:299] Negotiation successful
0807 00:44:37.595117 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260807 00:44:37.601332 1873 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:37.602088 1873 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:37.608613 1873 catalog_manager.cc:5868] T 00000000000000000000000000000000 P aa42c30817094621a34709a9d8490697: Generated new TSK 0
I20260807 00:44:37.608816 1873 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:44:37.629119 1796 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.627821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38388 (local address 127.1.116.188:43041)
0807 00:44:37.627868 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.627870 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.628505 (+ 635us) server_negotiation.cc:407] Connection header received
0807 00:44:37.628602 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.628603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.628614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.628645 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.629052 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38388: BlockingRecv error: recv got EOF from 127.0.0.1:38388 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 00:44:37.629799 1557 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.628227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46116 (local address 127.1.116.190:42541)
0807 00:44:37.628265 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.628267 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.629348 (+ 1081us) server_negotiation.cc:407] Connection header received
0807 00:44:37.629437 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.629438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.629447 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.629475 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.629742 (+ 267us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46116: BlockingRecv error: recv got EOF from 127.0.0.1:46116 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 00:44:37.630877 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.630154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.1.116.189:37065)
0807 00:44:37.630191 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.630193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.630324 (+ 131us) server_negotiation.cc:407] Connection header received
0807 00:44:37.630412 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.630413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.630422 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.630451 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.630822 (+ 371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47266: BlockingRecv error: recv got EOF from 127.0.0.1:47266 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260807 00:44:37.639539 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.638758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47270 (local address 127.1.116.189:37065)
0807 00:44:37.638793 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.638795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.638996 (+ 201us) server_negotiation.cc:407] Connection header received
0807 00:44:37.639102 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.639103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.639119 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.639148 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.639493 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47270: BlockingRecv error: recv got EOF from 127.0.0.1:47270 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 00:44:37.642572 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.641789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47278 (local address 127.1.116.189:37065)
0807 00:44:37.641821 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.641823 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.642036 (+ 213us) server_negotiation.cc:407] Connection header received
0807 00:44:37.642129 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.642130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.642140 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.642170 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.642509 (+ 339us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47278: BlockingRecv error: recv got EOF from 127.0.0.1:47278 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 00:44:37.646754 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.645899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47286 (local address 127.1.116.189:37065)
0807 00:44:37.645934 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.645936 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.646168 (+ 232us) server_negotiation.cc:407] Connection header received
0807 00:44:37.646273 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.646274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.646284 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.646318 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.646709 (+ 391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47286: BlockingRecv error: recv got EOF from 127.0.0.1:47286 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 00:44:37.652563 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.651831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47292 (local address 127.1.116.189:37065)
0807 00:44:37.651864 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.651866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.652086 (+ 220us) server_negotiation.cc:407] Connection header received
0807 00:44:37.652183 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.652184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.652194 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.652221 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.652524 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47292: BlockingRecv error: recv got EOF from 127.0.0.1:47292 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 00:44:37.662360 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.661649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47306 (local address 127.1.116.189:37065)
0807 00:44:37.661679 (+ 30us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.661681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.661877 (+ 196us) server_negotiation.cc:407] Connection header received
0807 00:44:37.661965 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.661965 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.661974 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.662014 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.662317 (+ 303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47306: BlockingRecv error: recv got EOF from 127.0.0.1:47306 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260807 00:44:37.680508 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.679636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47318 (local address 127.1.116.189:37065)
0807 00:44:37.679677 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.679679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.679921 (+ 242us) server_negotiation.cc:407] Connection header received
0807 00:44:37.680049 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.680050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.680060 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.680091 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.680458 (+ 367us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47318: BlockingRecv error: recv got EOF from 127.0.0.1:47318 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 00:44:37.714654 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.713799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47326 (local address 127.1.116.189:37065)
0807 00:44:37.713861 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.713863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.713987 (+ 124us) server_negotiation.cc:407] Connection header received
0807 00:44:37.714071 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.714072 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.714087 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.714167 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.714589 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47326: BlockingRecv error: recv got EOF from 127.0.0.1:47326 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260807 00:44:37.780680 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.779963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47328 (local address 127.1.116.189:37065)
0807 00:44:37.780066 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.780068 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.780134 (+ 66us) server_negotiation.cc:407] Connection header received
0807 00:44:37.780172 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.780173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.780185 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.780239 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.780581 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47328: BlockingRecv error: recv got EOF from 127.0.0.1:47328 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 00:44:37.911458 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:37.910263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47334 (local address 127.1.116.189:37065)
0807 00:44:37.910369 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:37.910372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:37.910475 (+ 103us) server_negotiation.cc:407] Connection header received
0807 00:44:37.910668 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:37.910669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:37.910684 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:37.910884 (+ 200us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:37.911353 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47334: BlockingRecv error: recv got EOF from 127.0.0.1:47334 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260807 00:44:38.169688 1679 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:38.168860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47344 (local address 127.1.116.189:37065)
0807 00:44:38.168959 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:38.168962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:38.169067 (+ 105us) server_negotiation.cc:407] Connection header received
0807 00:44:38.169169 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:38.169170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:38.169184 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:38.169234 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:38.169622 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47344: BlockingRecv error: recv got EOF from 127.0.0.1:47344 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:44:38.509483 1818 catalog_manager.cc:1259] Loaded cluster ID: b28d5b6519744ee184b9845a5a2805e6
I20260807 00:44:38.509594 1818 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: loading cluster ID for follower catalog manager: success
I20260807 00:44:38.510483 1818 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: acquiring CA information for follower catalog manager: success
I20260807 00:44:38.510989 1818 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eaf2626526e1442dba4d1d177789a0dc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:44:38.516846 1843 catalog_manager.cc:1259] Loaded cluster ID: b28d5b6519744ee184b9845a5a2805e6
I20260807 00:44:38.516911 1843 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: loading cluster ID for follower catalog manager: success
I20260807 00:44:38.517724 1843 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: acquiring CA information for follower catalog manager: success
I20260807 00:44:38.517970 1843 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b71b998e754c487ea8a1c50918af202f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:44:38.686131 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:38.683275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47354 (local address 127.1.116.189:37065)
0807 00:44:38.683491 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:38.683493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:38.683507 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:44:38.683541 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:38.683542 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:38.683600 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:38.683686 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:38.683867 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.684381 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.685298 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.685482 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.685539 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:38.685702 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:38.685822 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:38.685917 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:38.685980 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 00:44:38.685999 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":126,"threads_started":1}
I20260807 00:44:38.690809 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:38.688313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38390 (local address 127.1.116.188:43041)
0807 00:44:38.688534 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:38.688536 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:38.688550 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:44:38.688590 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:38.688591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:38.688625 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:38.688726 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:38.688818 (+ 92us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.689289 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.690068 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.690254 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.690326 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:38.690460 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:38.690606 (+ 146us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:38.690668 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:38.690734 (+ 66us) server_negotiation.cc:299] Negotiation successful
0807 00:44:38.690752 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":89,"threads_started":1}
I20260807 00:44:38.695776 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:38.692692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47366 (local address 127.1.116.189:37065)
0807 00:44:38.692731 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:38.692733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:38.692828 (+ 95us) server_negotiation.cc:407] Connection header received
0807 00:44:38.692946 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:38.692946 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:38.692959 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:38.692992 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:38.693169 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.693684 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.694604 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.694916 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.694995 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:38.695494 (+ 499us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:38.695544 (+ 50us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:38.695607 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:38.695687 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 00:44:38.695714 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:44:38.695776 2005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:38.692685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.1.116.188:43041)
0807 00:44:38.692733 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:38.692735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:38.693021 (+ 286us) server_negotiation.cc:407] Connection header received
0807 00:44:38.693147 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:38.693149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:38.693162 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:38.693219 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:38.693349 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.694105 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.695019 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.695217 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.695299 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:38.695456 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:38.695492 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:38.695552 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:38.695685 (+ 133us) server_negotiation.cc:299] Negotiation successful
0807 00:44:38.695713 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:44:38.696386 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:38.692736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46130 (local address 127.1.116.190:42541)
0807 00:44:38.692990 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:38.692993 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:38.693030 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:44:38.693129 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:38.693130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:38.693196 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:38.693310 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:38.693514 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.694249 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.695287 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:38.695593 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:38.695657 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:38.695888 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:38.695998 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:38.696049 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:38.696246 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 00:44:38.696288 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":86,"threads_started":1}
I20260807 00:44:38.698195 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1445
I20260807 00:44:38.715104 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1566
I20260807 00:44:38.726399 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 1685
2026-08-07T00:44:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3167 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:44:38 dist-test-slave-h6n0 krb5kdc[2021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:44:38 dist-test-slave-h6n0 krb5kdc[2021](info): set up 2 sockets
Aug 07 00:44:38 dist-test-slave-h6n0 krb5kdc[2021](info): commencing operation
krb5kdc: starting...
W20260807 00:44:39.974781 1490 mini_kdc.cc:121] Time spent starting KDC: real 1.222s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/test-user.keytab.
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:44:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:44:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.116.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
Entry for principal kudu/127.1.116.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab.
I20260807 00:44:40.122702 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39685
--webserver_interface=127.1.116.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39685,127.1.116.189:33753,127.1.116.188:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:44:40.215374 2042 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:44:40.215610 2042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:40.215627 2042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:40.215647 2042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:40.216706 2042 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:40.216747 2042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:40.216763 2042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:40.216775 2042 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:40.216787 2042 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:40.218267 2042 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39685,127.1.116.189:33753,127.1.116.188:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.190.keytab
--principal=kudu/127.1.116.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.190:39685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.116.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:40.218488 2042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:40.218701 2042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:40.224041 2054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:40.224057 2057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:40.224323 2050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:40.224524 2042 server_base.cc:1034] running on GCE node
I20260807 00:44:40.224817 2042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:40.225142 2042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:40.226320 2042 hybrid_clock.cc:648] HybridClock initialized: now 1786063480226304 us; error 50 us; skew 500 ppm
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:40.229280 2042 init.cc:377] Logged in from keytab as kudu/127.1.116.190@KRBTEST.COM (short username kudu)
I20260807 00:44:40.229816 2042 webserver.cc:460] Webserver started at http://127.1.116.190:32777/ using document root <none> and password file <none>
I20260807 00:44:40.230077 2042 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:40.230145 2042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:40.230295 2042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:40.231524 2042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data/instance:
uuid: "794b0052639a4ba3b802994fb21f6832"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "d1d07b063a2e5f3fb3c1aeb519d12af5"
server_key_iv: "0aa0b22831b9635c95880f436fb825ac"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.231911 2042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/wal/instance:
uuid: "794b0052639a4ba3b802994fb21f6832"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "d1d07b063a2e5f3fb3c1aeb519d12af5"
server_key_iv: "0aa0b22831b9635c95880f436fb825ac"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.233404 2042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260807 00:44:40.234280 2064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:40.234519 2042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:40.234669 2042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/wal
uuid: "794b0052639a4ba3b802994fb21f6832"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "d1d07b063a2e5f3fb3c1aeb519d12af5"
server_key_iv: "0aa0b22831b9635c95880f436fb825ac"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.234752 2042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:40.246908 2042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:40.249244 2042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:40.249411 2042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:40.254611 2042 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.190:39685
I20260807 00:44:40.254685 2140 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.190:39685 every 8 connection(s)
I20260807 00:44:40.255079 2042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/data/info.pb
I20260807 00:44:40.256222 2141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:40.258014 2141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:40.258814 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2042
I20260807 00:44:40.258924 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-0/wal/instance
I20260807 00:44:40.259192 1490 external_mini_cluster.cc:1428] Setting key fbfa512c1004751599eb849f33fb00df
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.189@KRBTEST.COM; defaulting to no policy
I20260807 00:44:40.270568 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.258822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39685 (local address 127.0.0.1:54366)
0807 00:44:40.259141 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.259194 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.259982 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.260340 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.260345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.260758 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.261024 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.261031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.262722 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.262723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.263902 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.263906 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.264494 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.264496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.264599 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.265181 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.265193 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.267009 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.269009 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.269016 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.269018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.269217 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.269477 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.269477 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.269478 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.269580 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.269941 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.269947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.270011 (+ 64us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.270116 (+ 105us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.270363 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"mutex_wait_us":44,"thread_start_us":107,"threads_started":1}
I20260807 00:44:40.270572 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.258917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54366 (local address 127.1.116.190:39685)
0807 00:44:40.259105 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.259113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.259355 (+ 242us) server_negotiation.cc:407] Connection header received
0807 00:44:40.260164 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.260166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.260185 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.260275 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.261162 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.262597 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.264006 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.264354 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.264453 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.267101 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.267109 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.267109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.267117 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.268920 (+ 1803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.269318 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.269321 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.269322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.269402 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.269644 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.269645 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.269646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.269870 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.269963 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.270159 (+ 196us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.270282 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260807 00:44:40.271587 2141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.189@KRBTEST.COM" created.
I20260807 00:44:40.272200 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.271867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34902)
0807 00:44:40.272086 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.272157 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
W20260807 00:44:40.272342 2065 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
I20260807 00:44:40.274894 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.274787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34904)
0807 00:44:40.274819 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.274862 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260807 00:44:40.275019 2141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:33753: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab.
Entry for principal kudu/127.1.116.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab.
I20260807 00:44:40.291241 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:33753
--webserver_interface=127.1.116.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab
--principal=kudu/127.1.116.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39685,127.1.116.189:33753,127.1.116.188:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:44:40.324226 2141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } attempt: 1
I20260807 00:44:40.324769 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.324541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34916)
0807 00:44:40.324636 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.324711 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 00:44:40.325207 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.325102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34928)
0807 00:44:40.325155 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.325186 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 00:44:40.325359 2141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:33753: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
W20260807 00:44:40.382781 2151 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:44:40.383049 2151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:40.383067 2151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:40.383090 2151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:40.384375 2151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:40.384429 2151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:40.384444 2151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:40.384459 2151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:40.384471 2151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:40.386179 2151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39685,127.1.116.189:33753,127.1.116.188:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.189.keytab
--principal=kudu/127.1.116.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.189:33753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.116.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:40.386499 2151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:40.386833 2151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:40.392520 2163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:40.392560 2165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:40.392612 2160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:40.392992 2151 server_base.cc:1034] running on GCE node
I20260807 00:44:40.393319 2151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:40.393522 2141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } attempt: 2
I20260807 00:44:40.393757 2151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:40.394166 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.393927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34934)
0807 00:44:40.394086 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.394129 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260807 00:44:40.394766 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.394621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34938)
0807 00:44:40.394666 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.394725 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260807 00:44:40.394944 2141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.189:33753: Network error: Client connection negotiation failed: client connection to 127.1.116.189:33753: connect: Connection refused (error 111)
I20260807 00:44:40.394989 2151 hybrid_clock.cc:648] HybridClock initialized: now 1786063480394946 us; error 40 us; skew 500 ppm
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:40.398343 2151 init.cc:377] Logged in from keytab as kudu/127.1.116.189@KRBTEST.COM (short username kudu)
I20260807 00:44:40.398977 2151 webserver.cc:460] Webserver started at http://127.1.116.189:44427/ using document root <none> and password file <none>
I20260807 00:44:40.399253 2151 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:40.399325 2151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:40.399502 2151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:40.401008 2151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data/instance:
uuid: "8e5851a1cf1f42578dd09be4d7d7186b"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "7884d312e125ba670fa8ea38cd4322b8"
server_key_iv: "3a21d5444677aabc53ba8bc89b32d4b2"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.401631 2151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/wal/instance:
uuid: "8e5851a1cf1f42578dd09be4d7d7186b"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "7884d312e125ba670fa8ea38cd4322b8"
server_key_iv: "3a21d5444677aabc53ba8bc89b32d4b2"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.403293 2151 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260807 00:44:40.404268 2172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:40.404578 2151 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:44:40.404736 2151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/wal
uuid: "8e5851a1cf1f42578dd09be4d7d7186b"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "7884d312e125ba670fa8ea38cd4322b8"
server_key_iv: "3a21d5444677aabc53ba8bc89b32d4b2"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.404829 2151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:40.416585 2151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:40.419224 2151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:40.419404 2151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:40.424121 2151 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.189:33753
I20260807 00:44:40.424172 2253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.189:33753 every 8 connection(s)
I20260807 00:44:40.424784 2151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/data/info.pb
I20260807 00:44:40.426047 2254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:40.427433 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2151
I20260807 00:44:40.427556 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-1/wal/instance
I20260807 00:44:40.427802 1490 external_mini_cluster.cc:1428] Setting key 52aef938cb0f904d2582c012e7690892
I20260807 00:44:40.428644 2254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.116.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.116.188@KRBTEST.COM" created.
I20260807 00:44:40.441747 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.429910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54380 (local address 127.1.116.190:39685)
0807 00:44:40.429995 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.429997 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.430280 (+ 283us) server_negotiation.cc:407] Connection header received
0807 00:44:40.430957 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.430958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.430978 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.431053 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.431947 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.432508 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.434346 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.434657 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.434738 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.438072 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.438074 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.438075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.438086 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.439924 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.440548 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.440550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.440550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.440611 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.440950 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.440950 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.440951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.441095 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.441156 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.441494 (+ 338us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.441604 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:44:40.441898 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.429806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39685 (local address 127.0.0.1:54380)
0807 00:44:40.430052 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.430111 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.430847 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.431145 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.431149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.431479 (+ 330us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.431769 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.431775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.432659 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.432660 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.434208 (+ 1548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.434212 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.434784 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.434786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.434998 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.435730 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.435745 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.437946 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.440044 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.440053 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.440056 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.440458 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.440689 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.440690 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.440691 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.440864 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.441170 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.441176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.441319 (+ 143us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.441450 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.441655 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":134,"threads_started":1}
I20260807 00:44:40.442662 2254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:44:40.451930 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.442999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34950)
0807 00:44:40.443054 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.443065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.443109 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.443427 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.443428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.443586 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.443642 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.443643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.444926 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.444927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.445503 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.445504 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.446205 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.446206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.446304 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.446905 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.446911 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.448371 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.450459 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.450460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.450461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.450709 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.450935 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.450935 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.450936 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.451073 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.451616 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.451617 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.451661 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.451734 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.451844 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260807 00:44:40.452080 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.443107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34950 (local address 127.1.116.189:33753)
0807 00:44:40.443301 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.443305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.443321 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:44:40.443336 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.443338 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.443348 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.443427 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.443725 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.444811 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.445633 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.446081 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.446199 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.448560 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.448571 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.448572 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.448588 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.450356 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.450799 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.450802 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.450803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.450861 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.451191 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.451192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.451193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.451482 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.451647 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.451767 (+ 120us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.451920 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":167,"thread_start_us":115,"threads_started":1}
I20260807 00:44:40.452833 2254 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:40.453318 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.453129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35592)
0807 00:44:40.453179 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.453289 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260807 00:44:40.453449 2174 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
I20260807 00:44:40.454006 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.453822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35600)
0807 00:44:40.453887 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.453927 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 00:44:40.454177 2254 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:46633: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.116.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab.
Entry for principal kudu/127.1.116.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab.
I20260807 00:44:40.461174 1490 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskLvHwjp/build/release/bin/kudu
/tmp/dist-test-taskLvHwjp/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:46633
--webserver_interface=127.1.116.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab
--principal=kudu/127.1.116.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.116.148:37361
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.116.190:39685,127.1.116.189:33753,127.1.116.188:46633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:44:40.486129 2141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } attempt: 3
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:44:40.493870 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.486766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34964 (local address 127.1.116.189:33753)
0807 00:44:40.486868 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.486871 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.486884 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:44:40.486924 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.486925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.486937 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.487013 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.487383 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.487927 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.488677 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.488914 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.489007 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.491046 (+ 2039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.491047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.491048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.491056 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.492531 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.492918 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.492919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.492920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.492990 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.493210 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.493211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.493212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.493413 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.493528 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.493692 (+ 164us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.493771 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:44:40.493870 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.486673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34964)
0807 00:44:40.486749 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.486762 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.486834 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.487021 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.487022 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.487211 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.487287 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.487289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.488022 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.488023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.488568 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.488569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.489007 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.489008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.489089 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.489614 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.489626 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.490950 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.492641 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.492642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.492642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.492828 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.493062 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.493071 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.493072 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.493141 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.493518 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.493519 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.493579 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.493666 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.493770 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 00:44:40.494766 2141 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:40.495160 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.495060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35610)
0807 00:44:40.495109 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.495142 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 00:44:40.495445 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.495358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35624)
0807 00:44:40.495404 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.495426 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 00:44:40.495532 2141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:46633: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
I20260807 00:44:40.503499 2254 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } attempt: 1
I20260807 00:44:40.504067 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.503940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35640)
0807 00:44:40.504007 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.504042 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260807 00:44:40.504691 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.504475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35648)
0807 00:44:40.504576 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.504650 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260807 00:44:40.504899 2254 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:46633: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
I20260807 00:44:40.526690 2141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } attempt: 1
I20260807 00:44:40.527417 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.527230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35650)
0807 00:44:40.527318 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.527374 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 00:44:40.527931 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.527816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35654)
0807 00:44:40.527873 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.527908 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260807 00:44:40.528211 2141 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:46633: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
W20260807 00:44:40.556912 2268 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:44:40.557166 2268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:44:40.557184 2268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:44:40.557236 2268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:44:40.558398 2268 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:44:40.558439 2268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:44:40.558454 2268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:44:40.558466 2268 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:44:40.558516 2268 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:44:40.560197 2268 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.116.148:37361
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.116.190:39685,127.1.116.189:33753,127.1.116.188:46633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/krb5kdc/kudu_127.1.116.188.keytab
--principal=kudu/127.1.116.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.116.188:46633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.116.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 23:47:59 UTC on ccycloud.quasar-wqpycd.root.comops.site
build id 4571632
I20260807 00:44:40.560482 2268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:44:40.560767 2268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:44:40.566169 2276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:40.566125 2278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:44:40.566151 2274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:44:40.566593 2268 server_base.cc:1034] running on GCE node
I20260807 00:44:40.566864 2268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:44:40.567139 2268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:44:40.568307 2268 hybrid_clock.cc:648] HybridClock initialized: now 1786063480568282 us; error 43 us; skew 500 ppm
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:44:40.571209 2268 init.cc:377] Logged in from keytab as kudu/127.1.116.188@KRBTEST.COM (short username kudu)
I20260807 00:44:40.571724 2268 webserver.cc:460] Webserver started at http://127.1.116.188:46753/ using document root <none> and password file <none>
I20260807 00:44:40.571991 2268 fs_manager.cc:359] Metadata directory not provided
I20260807 00:44:40.572058 2268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:44:40.572213 2268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:44:40.573089 2254 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } attempt: 2
I20260807 00:44:40.573568 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.573399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35670)
0807 00:44:40.573480 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.573525 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 00:44:40.573501 2268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data/instance:
uuid: "9e835c10c7294ffba15cf8664e141717"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "8c5983c7cf7c18704a3b7f2a52174fd9"
server_key_iv: "0b8b50f5eaf71fbba9d4cb1f1b5c4025"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.573925 2268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/wal/instance:
uuid: "9e835c10c7294ffba15cf8664e141717"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "8c5983c7cf7c18704a3b7f2a52174fd9"
server_key_iv: "0b8b50f5eaf71fbba9d4cb1f1b5c4025"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.574128 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.573951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35676)
0807 00:44:40.574060 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.574107 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260807 00:44:40.574301 2254 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.116.188:46633: Network error: Client connection negotiation failed: client connection to 127.1.116.188:46633: connect: Connection refused (error 111)
I20260807 00:44:40.575635 2268 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260807 00:44:40.576552 2289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:44:40.576788 2268 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:44:40.576894 2268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data,/tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/wal
uuid: "9e835c10c7294ffba15cf8664e141717"
format_stamp: "Formatted at 2026-08-07 00:44:40 on dist-test-slave-h6n0"
server_key: "8c5983c7cf7c18704a3b7f2a52174fd9"
server_key_iv: "0b8b50f5eaf71fbba9d4cb1f1b5c4025"
server_key_version: "encryptionkey@0"
I20260807 00:44:40.576972 2268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:44:40.589038 2268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:44:40.591285 2268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:44:40.591449 2268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:44:40.596765 2268 rpc_server.cc:307] RPC server started. Bound to: 127.1.116.188:46633
I20260807 00:44:40.596814 2370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.116.188:46633 every 8 connection(s)
I20260807 00:44:40.597267 2268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/data/info.pb
I20260807 00:44:40.598318 1490 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskLvHwjp/build/release/bin/kudu as pid 2268
I20260807 00:44:40.598443 1490 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskLvHwjp/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786063190385254-1490-0/minicluster-data/master-2/wal/instance
I20260807 00:44:40.598634 2371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:44:40.598752 1490 external_mini_cluster.cc:1428] Setting key a673a9ede556325a60115500783d65f3
I20260807 00:44:40.601215 2371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:40.603401 2141 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } attempt: 2
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.190@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:44:40.612124 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.601950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.1.116.190:39685)
0807 00:44:40.602047 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.602050 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.602228 (+ 178us) server_negotiation.cc:407] Connection header received
0807 00:44:40.602318 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.602319 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.602329 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.602383 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.603020 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.603584 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.604857 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.605122 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.605165 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.608425 (+ 3260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.608427 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.608427 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.608435 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.610250 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.610899 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.610901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.610902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.610973 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.611271 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.611272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.611273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.611523 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.611601 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.611907 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.612014 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:44:40.612287 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.601896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.190:39685 (local address 127.0.0.1:54396)
0807 00:44:40.602108 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.602151 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.602243 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.602425 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.602428 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.602816 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.602920 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.602924 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.603712 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.603713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.604749 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.604753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.605300 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.605302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.605407 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.605934 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.605946 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.608294 (+ 2348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.610360 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.610367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.610370 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.610837 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.611040 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.611041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.611041 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.611178 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.611647 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.611654 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.611736 (+ 82us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.611848 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.612026 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":124,"threads_started":1}
I20260807 00:44:40.612314 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.603882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.1.116.188:46633)
0807 00:44:40.604035 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.604037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.604054 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:44:40.604072 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.604072 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.604083 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.604157 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.604518 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.605227 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.606105 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.606340 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.606414 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.608598 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.608608 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.608609 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.608616 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.610740 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.611153 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.611156 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.611156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.611221 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.611460 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.611462 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.611462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.611708 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.612040 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.612090 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.612191 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260807 00:44:40.612953 2371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:44:40.613056 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.599367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35682 (local address 127.1.116.188:46633)
0807 00:44:40.599605 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.599616 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.599655 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:44:40.600474 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.600484 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.600559 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.600723 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.601535 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.602925 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.603944 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.604467 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.604624 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.608636 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.608638 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.608638 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.608645 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.611329 (+ 2684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.611730 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.611731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.611732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.611787 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.612049 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.612050 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.612050 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.612164 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.612792 (+ 628us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.612835 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.612926 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":109,"threads_started":1}
I20260807 00:44:40.613267 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.603679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35692)
0807 00:44:40.603751 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.603764 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.603811 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.604190 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.604191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.604381 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.604425 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.604426 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.605326 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.605327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.606015 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.606017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.606413 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.606414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.606471 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.606935 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.606940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.608433 (+ 1493us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.610832 (+ 2399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.610834 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.610834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.611064 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.611301 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.611302 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.611302 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.611379 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.611804 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.611805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.611857 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.611911 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.613173 (+ 1262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":1111}
I20260807 00:44:40.614823 2141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832: Bootstrap starting.
I20260807 00:44:40.615600 2141 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:40.615830 2141 log.cc:824] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:40.617183 2141 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832: No bootstrap required, opened a new log
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:44:40.619380 2141 raft_consensus.cc:348] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
I20260807 00:44:40.619540 2141 raft_consensus.cc:729] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 794b0052639a4ba3b802994fb21f6832, State: Initialized, Role: FOLLOWER
I20260807 00:44:40.619623 2141 consensus_queue.cc:260] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
I20260807 00:44:40.620009 2141 sys_catalog.cc:564] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:40.619949 2387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.620117 2387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:40.621385 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.613231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.189:33753 (local address 127.0.0.1:34972)
0807 00:44:40.613552 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.613564 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.613615 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.613763 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.613764 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.613909 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.613965 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.613966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.614638 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.614640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.615254 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.615256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.615663 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.615664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.615739 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.616183 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.616189 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.617755 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.619844 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.619845 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.619846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.620341 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.620604 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.620605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.620605 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.620677 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.620993 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.620994 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.621046 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.621117 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.621258 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308}
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.190@KRBTEST.COM
I20260807 00:44:40.621726 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.613287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34972 (local address 127.1.116.189:33753)
0807 00:44:40.613358 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.613361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.613629 (+ 268us) server_negotiation.cc:407] Connection header received
0807 00:44:40.613667 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.613668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.613682 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.613744 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.614054 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.614522 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.615353 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.615589 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.615665 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.617881 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.617883 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.617883 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.617892 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.619739 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.620436 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.620437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.620438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.620517 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.620776 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.620777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.620777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.620919 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.621002 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.621530 (+ 528us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.621619 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 00:44:40.622280 2371 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } has no permanent_uuid. Determining permanent_uuid...
W20260807 00:44:40.623545 2407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:40.623612 2407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:40.625766 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.614765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54400 (local address 127.1.116.190:39685)
0807 00:44:40.614803 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.614805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.616121 (+ 1316us) server_negotiation.cc:407] Connection header received
0807 00:44:40.616206 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.616207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.616218 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.616265 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.616904 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.617656 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.618521 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.618752 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.618845 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.622380 (+ 3535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.622381 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.622382 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.622389 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.624207 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.624605 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.624607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.624607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.624667 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.624871 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.624872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.624872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.624999 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.625172 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.625605 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.625687 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260807 00:44:40.623989 2042 master_runner.cc:186] Error getting master registration for 127.1.116.189:33753: OK, instance_id { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" instance_seqno: 1786063480422772 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:44:40.627023 2042 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.188@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:44:40.631508 2379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.622519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35704)
0807 00:44:40.622681 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.622692 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.622750 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.623211 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.623212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.623945 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.623980 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.623982 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.624868 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.624869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.625549 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.625552 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.626010 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.626011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.626052 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.626566 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.626571 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.628013 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.629863 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.629864 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.629864 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.630632 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.630842 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.630842 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.630843 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.630899 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.631160 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.631161 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.631202 (+ 41us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.631278 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.631400 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260807 00:44:40.631841 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.622619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35704 (local address 127.1.116.188:46633)
0807 00:44:40.622659 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.622661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.623070 (+ 409us) server_negotiation.cc:407] Connection header received
0807 00:44:40.623093 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.623094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.623104 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.623162 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.624073 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.624765 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.625649 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.625936 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.626210 (+ 274us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.628120 (+ 1910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.628121 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.628122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.628128 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.629769 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.630715 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.630716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.630716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.630781 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.630969 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.630970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.630970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.631095 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.631167 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.631314 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.631462 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":21}
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.116.189@KRBTEST.COM
I20260807 00:44:40.633662 2371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: Bootstrap starting.
I20260807 00:44:40.634379 2371 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:40.634737 2371 log.cc:824] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:40.635722 2371 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: No bootstrap required, opened a new log
I20260807 00:44:40.635921 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.627578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34984 (local address 127.1.116.189:33753)
0807 00:44:40.627621 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.627622 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.627631 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:44:40.627685 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.627686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.627696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.627745 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.628251 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.628873 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.629692 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.629971 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.630046 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.632790 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.632791 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.632791 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.632799 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.634619 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.635056 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.635057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.635057 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.635111 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.635320 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.635321 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.635321 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.635466 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.635605 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.635752 (+ 147us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.635833 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260807 00:44:40.637610 1490 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:44:40.637724 2371 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
I20260807 00:44:40.637826 2371 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e835c10c7294ffba15cf8664e141717, State: Initialized, Role: FOLLOWER
I20260807 00:44:40.637874 2371 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
I20260807 00:44:40.638186 2371 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:40.638100 2411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.638247 2411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:40.638854 2104 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "794b0052639a4ba3b802994fb21f6832"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54400
I20260807 00:44:40.639016 2104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 00:44:40.639067 2104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:40.640113 2104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
W20260807 00:44:40.640192 2430 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:40.640242 2430 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:40.640388 2104 leader_election.cc:288] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [CANDIDATE]: Term 1 election: Requested vote from peers 8e5851a1cf1f42578dd09be4d7d7186b (127.1.116.189:33753), 9e835c10c7294ffba15cf8664e141717 (127.1.116.188:46633)
I20260807 00:44:40.640924 2326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "794b0052639a4ba3b802994fb21f6832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9e835c10c7294ffba15cf8664e141717"
I20260807 00:44:40.641032 2326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:40.641870 2326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 794b0052639a4ba3b802994fb21f6832 in term 1.
I20260807 00:44:40.642127 2065 leader_election.cc:302] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 794b0052639a4ba3b802994fb21f6832, 9e835c10c7294ffba15cf8664e141717; no voters:
I20260807 00:44:40.642273 2387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:44:40.642442 2387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [term 1 LEADER]: Becoming Leader. State: Replica: 794b0052639a4ba3b802994fb21f6832, State: Running, Role: LEADER
I20260807 00:44:40.642573 2387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
I20260807 00:44:40.643016 2387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 794b0052639a4ba3b802994fb21f6832. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.643111 2387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:40.644083 2447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:44:40.644861 2447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:44:40.643764 2268 master_runner.cc:186] Error getting master registration for 127.1.116.189:33753: OK, instance_id { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" instance_seqno: 1786063480422772 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:44:40.646449 2268 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:44:40.648232 2326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [term 1 FOLLOWER]: Refusing update from remote peer 794b0052639a4ba3b802994fb21f6832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:40.648747 2387 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:40.649993 2411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 794b0052639a4ba3b802994fb21f6832. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.650105 2411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:40.654995 2145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.643369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54410 (local address 127.1.116.190:39685)
0807 00:44:40.643502 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.643504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.646574 (+ 3070us) server_negotiation.cc:407] Connection header received
0807 00:44:40.646743 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.646744 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.646756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.646819 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.647597 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.648375 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.649648 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.649949 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.649990 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.651842 (+ 1852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.651844 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.651844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.651851 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.653620 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.654057 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.654059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.654059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.654115 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.654341 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.654342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.654342 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.654469 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.654592 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.654817 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.654898 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260807 00:44:40.655732 2387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.655826 2387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: This master's current role is: LEADER
I20260807 00:44:40.656416 2411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.656514 2411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:40.656975 2452 mvcc.cc:205] Tried to move back new op lower bound from 7315716016730705920 to 7315716016712822784. Current Snapshot: MvccSnapshot[applied={T|T < 7315716016730705920}]
I20260807 00:44:40.661983 2447 catalog_manager.cc:1347] Generated new cluster ID: 2ead69996cb540b2a09c1463ae061532
I20260807 00:44:40.662034 2447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:44:40.665447 2254 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } attempt: 3
Aug 07 00:44:40 dist-test-slave-h6n0 krb5kdc[2021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786063480, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.116.189@KRBTEST.COM for kudu/127.1.116.188@KRBTEST.COM
I20260807 00:44:40.674399 2259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.665712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.116.188:46633 (local address 127.0.0.1:35722)
0807 00:44:40.665762 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0807 00:44:40.665771 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0807 00:44:40.665805 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:44:40.666074 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:44:40.666075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:44:40.666236 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:44:40.666279 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.666281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.667290 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.667291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.668117 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:44:40.668119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.668641 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.668642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:44:40.668697 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.669105 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:44:40.669109 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:44:40.670659 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:44:40.673261 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.673262 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.673263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.673479 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.673701 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:44:40.673702 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:44:40.673702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:44:40.673762 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:44:40.674091 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:44:40.674092 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:44:40.674138 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 00:44:40.674215 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 00:44:40.674302 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260807 00:44:40.674834 2381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.665883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.1.116.188:46633)
0807 00:44:40.665949 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.665951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.665959 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:44:40.665983 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.665984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.665996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.666065 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.666379 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.667208 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.668210 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.668567 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.668644 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.670755 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.670757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.670757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.670764 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.673171 (+ 2407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.673566 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.673567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.673568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.673631 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.673863 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.673863 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.673864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.674024 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.674097 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.674634 (+ 537us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.674745 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 00:44:40.676640 2254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: Bootstrap starting.
I20260807 00:44:40.677525 2254 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: Neither blocks nor log segments found. Creating new log.
I20260807 00:44:40.677851 2254 log.cc:824] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:44:40.682932 2376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.643428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.1.116.188:46633)
0807 00:44:40.643539 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.643542 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.650581 (+ 7039us) server_negotiation.cc:407] Connection header received
0807 00:44:40.650829 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.650830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.650854 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.650885 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.657564 (+ 6679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.658279 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.659325 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.659645 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.659721 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.661656 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.661657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.661658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.661664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.663970 (+ 2306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.682017 (+ 18047us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.682018 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.682018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.682088 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.682297 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.682297 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.682298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.682422 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.682506 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.682758 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.682845 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260807 00:44:40.684211 2263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:40.643579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34996 (local address 127.1.116.189:33753)
0807 00:44:40.643643 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.643646 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.650482 (+ 6836us) server_negotiation.cc:407] Connection header received
0807 00:44:40.650973 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.650974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.650986 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.651016 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.657558 (+ 6542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.658307 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.660198 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:40.660840 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:40.660908 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:40.679891 (+ 18983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:44:40.679893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:44:40.679893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:44:40.679901 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:44:40.681753 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.682689 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.682691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.682691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.682763 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:44:40.683349 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:44:40.683363 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:44:40.683364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:44:40.683517 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:44:40.683607 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:40.683996 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 00:44:40.684095 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 00:44:40.685257 2254 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: No bootstrap required, opened a new log
I20260807 00:44:40.687426 2254 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
I20260807 00:44:40.687584 2254 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e5851a1cf1f42578dd09be4d7d7186b, State: Initialized, Role: FOLLOWER
I20260807 00:44:40.687673 2254 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } }
I20260807 00:44:40.688033 2254 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:44:40.688153 2212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:44:40.689309 2212 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [term 1 FOLLOWER]: Refusing update from remote peer 794b0052639a4ba3b802994fb21f6832: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:44:40.689965 2213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "794b0052639a4ba3b802994fb21f6832" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8e5851a1cf1f42578dd09be4d7d7186b"
I20260807 00:44:40.690115 2441 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:44:40.690940 2213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 794b0052639a4ba3b802994fb21f6832 in term 1.
I20260807 00:44:40.694900 2447 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:44:40.695648 2447 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:44:40.693082 2468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.698774 2468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:40.698894 2468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 794b0052639a4ba3b802994fb21f6832. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.699007 2468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:40.699080 2468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.699195 2468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:44:40.700012 2481 mvcc.cc:205] Tried to move back new op lower bound from 7315716016730705920 to 7315716016712822784. Current Snapshot: MvccSnapshot[applied={T|T < 7315716016730705920}]
I20260807 00:44:40.700903 2441 sys_catalog.cc:455] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "794b0052639a4ba3b802994fb21f6832" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "794b0052639a4ba3b802994fb21f6832" member_type: VOTER last_known_addr { host: "127.1.116.190" port: 39685 } } peers { permanent_uuid: "8e5851a1cf1f42578dd09be4d7d7186b" member_type: VOTER last_known_addr { host: "127.1.116.189" port: 33753 } } peers { permanent_uuid: "9e835c10c7294ffba15cf8664e141717" member_type: VOTER last_known_addr { host: "127.1.116.188" port: 46633 } } }
I20260807 00:44:40.701030 2441 sys_catalog.cc:458] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832 [sys.catalog]: This master's current role is: LEADER
W20260807 00:44:40.701392 2486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:44:40.701493 2486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:44:40.702286 2447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 794b0052639a4ba3b802994fb21f6832: Generated new TSK 0
I20260807 00:44:40.702471 2447 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:44:40.725003 2145 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.723341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.1.116.190:39685)
0807 00:44:40.723390 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.723393 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.724338 (+ 945us) server_negotiation.cc:407] Connection header received
0807 00:44:40.724448 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.724449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.724462 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.724492 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.724929 (+ 437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54416: BlockingRecv error: recv got EOF from 127.0.0.1:54416 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 00:44:40.725968 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.723721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35002 (local address 127.1.116.189:33753)
0807 00:44:40.723761 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.723763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.725457 (+ 1694us) server_negotiation.cc:407] Connection header received
0807 00:44:40.725552 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.725553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.725565 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.725596 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.725908 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35002: BlockingRecv error: recv got EOF from 127.0.0.1:35002 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260807 00:44:40.726873 2376 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.724012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.1.116.188:46633)
0807 00:44:40.724056 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.724058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.726338 (+ 2280us) server_negotiation.cc:407] Connection header received
0807 00:44:40.726427 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.726428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.726440 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.726468 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.726809 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35736: BlockingRecv error: recv got EOF from 127.0.0.1:35736 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260807 00:44:40.729346 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.728595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35010 (local address 127.1.116.189:33753)
0807 00:44:40.728634 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.728635 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.728820 (+ 185us) server_negotiation.cc:407] Connection header received
0807 00:44:40.728923 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.728924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.728943 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.728970 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.729301 (+ 331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35010: BlockingRecv error: recv got EOF from 127.0.0.1:35010 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260807 00:44:40.732542 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.731790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35024 (local address 127.1.116.189:33753)
0807 00:44:40.731824 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.731826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.732059 (+ 233us) server_negotiation.cc:407] Connection header received
0807 00:44:40.732158 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.732159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.732170 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.732198 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.732500 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35024: BlockingRecv error: recv got EOF from 127.0.0.1:35024 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 00:44:40.736692 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.735871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35028 (local address 127.1.116.189:33753)
0807 00:44:40.735915 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.735917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.736156 (+ 239us) server_negotiation.cc:407] Connection header received
0807 00:44:40.736260 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.736261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.736273 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.736305 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.736646 (+ 341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35028: BlockingRecv error: recv got EOF from 127.0.0.1:35028 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260807 00:44:40.747442 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.742974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.1.116.189:33753)
0807 00:44:40.743016 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.743018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.746755 (+ 3737us) server_negotiation.cc:407] Connection header received
0807 00:44:40.746879 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.746880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.746895 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.746928 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.747378 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35032: BlockingRecv error: recv got EOF from 127.0.0.1:35032 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260807 00:44:40.758162 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.757129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35034 (local address 127.1.116.189:33753)
0807 00:44:40.757191 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.757194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.757479 (+ 285us) server_negotiation.cc:407] Connection header received
0807 00:44:40.757593 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.757594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.757608 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.757641 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.758083 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35034: BlockingRecv error: recv got EOF from 127.0.0.1:35034 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260807 00:44:40.777392 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.776196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.1.116.189:33753)
0807 00:44:40.776257 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.776259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.776613 (+ 354us) server_negotiation.cc:407] Connection header received
0807 00:44:40.776751 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.776753 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.776770 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.776810 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.777314 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35046: BlockingRecv error: recv got EOF from 127.0.0.1:35046 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260807 00:44:40.812485 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.811311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35060 (local address 127.1.116.189:33753)
0807 00:44:40.811379 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.811382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.811727 (+ 345us) server_negotiation.cc:407] Connection header received
0807 00:44:40.811844 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.811846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.811861 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.811906 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.812412 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35060: BlockingRecv error: recv got EOF from 127.0.0.1:35060 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260807 00:44:40.879621 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:40.878585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35066 (local address 127.1.116.189:33753)
0807 00:44:40.878653 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:40.878655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:40.878832 (+ 177us) server_negotiation.cc:407] Connection header received
0807 00:44:40.878878 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:40.878879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:40.878895 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:40.878972 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:40.879546 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35066: BlockingRecv error: recv got EOF from 127.0.0.1:35066 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
W20260807 00:44:41.010035 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:41.009062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35080 (local address 127.1.116.189:33753)
0807 00:44:41.009150 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:41.009153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:41.009369 (+ 216us) server_negotiation.cc:407] Connection header received
0807 00:44:41.009435 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:41.009436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:41.009450 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:41.009512 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:41.009941 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35080: BlockingRecv error: recv got EOF from 127.0.0.1:35080 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 00:44:41.268339 2263 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:44:41.267546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35092 (local address 127.1.116.189:33753)
0807 00:44:41.267634 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:41.267637 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:41.267654 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:44:41.267740 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:41.267741 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:41.267753 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:41.267843 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:44:41.268231 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35092: BlockingRecv error: recv got EOF from 127.0.0.1:35092 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:44:41.640669 2430 catalog_manager.cc:1259] Loaded cluster ID: 2ead69996cb540b2a09c1463ae061532
I20260807 00:44:41.640761 2430 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: loading cluster ID for follower catalog manager: success
I20260807 00:44:41.641579 2430 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: acquiring CA information for follower catalog manager: success
I20260807 00:44:41.641889 2430 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9e835c10c7294ffba15cf8664e141717: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:44:41.702069 2486 catalog_manager.cc:1259] Loaded cluster ID: 2ead69996cb540b2a09c1463ae061532
I20260807 00:44:41.702209 2486 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: loading cluster ID for follower catalog manager: success
I20260807 00:44:41.703277 2486 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: acquiring CA information for follower catalog manager: success
I20260807 00:44:41.703588 2486 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8e5851a1cf1f42578dd09be4d7d7186b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:44:41.784736 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:41.781843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.1.116.189:33753)
0807 00:44:41.782022 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:41.782024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:41.782038 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:44:41.782110 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:41.782110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:41.782173 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:41.782260 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:41.782432 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.783010 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.783848 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.784047 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.784110 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:41.784346 (+ 236us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:41.784456 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:41.784534 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:41.784604 (+ 70us) server_negotiation.cc:299] Negotiation successful
0807 00:44:41.784623 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":76,"threads_started":1}
I20260807 00:44:41.790241 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:41.787052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35750 (local address 127.1.116.188:46633)
0807 00:44:41.787243 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:41.787245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:41.787253 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:44:41.787300 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:41.787301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:41.787347 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:41.787428 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:41.787589 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.788289 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.789160 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.789465 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.789532 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:41.789686 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:41.789806 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:41.789887 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:41.790108 (+ 221us) server_negotiation.cc:299] Negotiation successful
0807 00:44:41.790140 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":91,"threads_started":1}
I20260807 00:44:41.795794 2588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:41.792634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35108 (local address 127.1.116.189:33753)
0807 00:44:41.792684 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:41.792686 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:41.792733 (+ 47us) server_negotiation.cc:407] Connection header received
0807 00:44:41.792854 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:41.792855 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:41.792866 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:41.792898 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:41.793190 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.793696 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.794899 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.795197 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.795263 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:41.795458 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:41.795502 (+ 44us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:41.795554 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:41.795672 (+ 118us) server_negotiation.cc:299] Negotiation successful
0807 00:44:41.795704 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260807 00:44:41.795837 2598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:41.792638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35754 (local address 127.1.116.188:46633)
0807 00:44:41.792731 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:41.792733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:41.792772 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:44:41.792846 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:41.792847 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:41.792859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:41.792926 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:41.793091 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.793828 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.795049 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.795271 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.795362 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:41.795520 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:41.795554 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:41.795596 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:41.795706 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 00:44:41.795733 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:44:41.796108 2608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:44:41.792357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54432 (local address 127.1.116.190:39685)
0807 00:44:41.792473 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:44:41.792475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0807 00:44:41.792529 (+ 54us) server_negotiation.cc:407] Connection header received
0807 00:44:41.792700 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:44:41.792701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:44:41.792754 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:44:41.792843 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:44:41.793140 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.793893 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.794978 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:44:41.795299 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:44:41.795356 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:44:41.795580 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:44:41.795699 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:44:41.795760 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:44:41.796011 (+ 251us) server_negotiation.cc:299] Negotiation successful
0807 00:44:41.796034 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":61,"threads_started":1}
I20260807 00:44:41.798367 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2042
I20260807 00:44:41.812350 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2151
I20260807 00:44:41.822319 1490 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskLvHwjp/build/release/bin/kudu with pid 2268
2026-08-07T00:44:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3098 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6265 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291442 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST